feat: support COSI BucketInfo in replica config #537
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
readBucketInfo
in theConfig
struct to read COSI BucketInfo for each database replica.parseBucketInfo
to handle the parsing of BucketInfo from a specified file.ReadConfigFile
function to call the newreadBucketInfo
method.litestream
by supporting COSI BucketInfo, enabling more versatile storage configurations in Kubernetes.Newly introduced code allows specifying new field called
bucket-info
. Configuration file as this:Will be expanded based on the contents of the BucketInfo file to something like this:
fix: remove deprecated code
fix: removed issues found by staticcheck.
feature: report errors from stopping/starting replication
COSI stands for Container Object Storage Interface. It is a standard developed by the Kubernetes community as part of the SIG-Storage project. COSI focuses specifically on defining an interface for object storage systems within the context of containerized applications and orchestrators like Kubernetes.
The primary goal of COSI is to provide a standardized way for containerized applications to interact with different object storage systems seamlessly. This interface allows developers and operators to use various object storage solutions without requiring changes to the application code. It abstracts the underlying details of the object storage provider, making it easier to switch between different storage backends.
Key components of COSI include:
BucketInfo: In the context of
litestream
's changes,BucketInfo
refers to the metadata or configuration information associated with a bucket in an object storage system. It specifies details such as the bucket name, authentication credentials, endpoint, and other relevant settings.The BucketInfo is a JSON object inside Secret, that can be mounted in the pod. In the mount path the new file will be created called
BucketInfo
. The contents should look like this:Protocols: COSI supports 3 different protocols for communication with object storage systems - S3 (Simple Storage Service), Azure Blob, GCS (Google Cloud Storage).
Authentication Types: COSI supports different authentication mechanisms, such as access keys, secret keys, and various authentication types specific to different object storage providers.
In the context of your
litestream
changes, it seems that support for parsing COSI BucketInfo is being added, allowinglitestream
to work with Kubernetes-compliant object storage configurations. This enhances the flexibility oflitestream
by supporting a wider range of object storage solutions through the COSI standard.