Deuxfleurs/garage

Fork 73

COSI Driver for garage #834

New issue

Open

opened 2024-06-06 15:42:11 +00:00 by shanduur · 2 comments

shanduur commented

2024-06-06 15:42:11 +00:00

Request:

I am requesting the implementation of a COSI driver for Garage. This implementation would enable seamless integration of Garage object storage with Kubernetes environments, allowing users to manage and consume Garage object storage using Kubernetes-native tools and APIs.

References:

COSI Specification:
- Kubernetes Container Object Storage Interface Specification
Existing Drivers:
Article on COSI:
- Kubernetes Blog: COSI - Kubernetes Object Storage Management

Benefits:

Standardization: Adopting COSI will provide a standardized way to manage and consume Garage object storage in Kubernetes, reducing the complexity of using proprietary interfaces.
Ecosystem Integration: By implementing a COSI driver, Garage will be more integrated into the Kubernetes ecosystem, enhancing its appeal to organizations using Kubernetes.
User Experience: Kubernetes users will benefit from a more consistent and simplified experience when working with Garage object storage, leveraging native Kubernetes APIs and tools.

Implementation Considerations:

Compatibility: Ensure the COSI driver for Garage is compatible with existing Kubernetes versions and aligns with the COSI specification.
Documentation: Provide comprehensive documentation and examples to guide users in setting up and using the COSI driver with Garage.
Testing: Implement thorough testing to ensure reliability and performance of the COSI driver in various Kubernetes environments.

Conclusion:

Implementing a COSI driver for Garage will greatly enhance its utility and adoption in Kubernetes-centric environments. It will provide a standardized, efficient, and user-friendly way to manage object storage, aligning Garage with modern cloud-native storage practices.

Meetings and Discussions:

Zoom Meeting:
- Join the meeting
Slack Discussion:
- Join Slack
Google Document with COSI meeting notes:
- Read the document

Thank you for considering this feature request.

As the adoption of Kubernetes continues to rise, the need for standardized interfaces to manage and consume different types of storage is becoming more critical. The Container Object Storage Interface (COSI) is a standard that aims to expose object storage to containerized workloads running in Kubernetes clusters, simplifying storage management and improving portability. **Request:** I am requesting the implementation of a COSI driver for Garage. This implementation would enable seamless integration of Garage object storage with Kubernetes environments, allowing users to manage and consume Garage object storage using Kubernetes-native tools and APIs. **References:** 1. **COSI Specification:** - [Kubernetes Container Object Storage Interface Specification](https://github.com/kubernetes-sigs/container-object-storage-interface-spec) 2. **Existing Drivers:** - [Azure COSI Driver](https://github.com/Azure/azure-cosi-driver) - [Ceph COSI](https://github.com/ceph/ceph-cosi) - [Linode COSI Driver](https://github.com/linode/linode-cosi-driver) - [Dell COSI](https://github.com/dell/cosi) - [S3GW COSI Driver](https://github.com/s3gw-tech/s3gw-cosi-driver) 3. **Article on COSI:** - [Kubernetes Blog: COSI - Kubernetes Object Storage Management](https://kubernetes.io/blog/2022/09/02/cosi-kubernetes-object-storage-management/) **Benefits:** - **Standardization:** Adopting COSI will provide a standardized way to manage and consume Garage object storage in Kubernetes, reducing the complexity of using proprietary interfaces. - **Ecosystem Integration:** By implementing a COSI driver, Garage will be more integrated into the Kubernetes ecosystem, enhancing its appeal to organizations using Kubernetes. - **User Experience:** Kubernetes users will benefit from a more consistent and simplified experience when working with Garage object storage, leveraging native Kubernetes APIs and tools. **Implementation Considerations:** - **Compatibility:** Ensure the COSI driver for Garage is compatible with existing Kubernetes versions and aligns with the COSI specification. - **Documentation:** Provide comprehensive documentation and examples to guide users in setting up and using the COSI driver with Garage. - **Testing:** Implement thorough testing to ensure reliability and performance of the COSI driver in various Kubernetes environments. **Conclusion:** Implementing a COSI driver for Garage will greatly enhance its utility and adoption in Kubernetes-centric environments. It will provide a standardized, efficient, and user-friendly way to manage object storage, aligning Garage with modern cloud-native storage practices. **Meetings and Discussions:** 1. **Zoom Meeting:** - [Join the meeting](https://zoom.us/j/614261834?pwd=Sk1USmtjR2t0MUdjTGVZeVVEV1BPQT09) 2. **Slack Discussion:** - [Join Slack](https://kubernetes.slack.com/archives/C017EGC1C6N) 3. **Google Document with COSI meeting notes:** - [Read the document](https://docs.google.com/document/d/1ZgEyyq3kk9BXP0bJL1Ki0JeOLILilGWZ4Vu2U4gs_QY/edit?usp=sharing) Thank you for considering this feature request.

👍 1

maximilien commented

2024-07-24 21:41:30 +00:00

Owner

Hello, thanks for the proposal. If I follow the specification, the CRDs are

BucketClaim - Represents a claim to provision a Bucket
BucketClass - Represents a class of Buckets with similar characteristics
Bucket - Represents a Bucket or its equivalent in the storage backend
BucketAccessClass - Represents a class of accessors with similar access requirements and authentication type
BucketAccess - Represents a resource to access a Bucket

I infer that this should not be unreasonable work, as we only need a thin wrapper to cater for the hooks related to

create a bucket
delete a bucket
create an access token
delete an access token
modify the permission of an access token

@shanduur any reasons why this driver should live in-tree? Most CSI drivers live outside of their storage backend codebase. It is likely easier to create a thin wrapper in go aside of garage than to do this in Rust.

Hello, thanks for the proposal. If I follow the specification, the CRDs are - BucketClaim - Represents a claim to provision a Bucket - BucketClass - Represents a class of Buckets with similar characteristics - Bucket - Represents a Bucket or its equivalent in the storage backend - BucketAccessClass - Represents a class of accessors with similar access requirements and authentication type - BucketAccess - Represents a resource to access a Bucket I infer that this should not be unreasonable work, as we only need a thin wrapper to cater for the hooks related to - create a bucket - delete a bucket - create an access token - delete an access token - modify the permission of an access token @shanduur any reasons why this driver should live in-tree? Most CSI drivers live outside of their storage backend codebase. It is likely easier to create a thin wrapper in go aside of garage than to do this in Rust.

maximilien added the

kind

ideas

label 2024-07-24 21:46:40 +00:00

quentin added the

labels 2024-08-07 10:14:48 +00:00

quentin changed title from ~~[Feature request] COSI Driver for garage~~ to COSI Driver for garage

2024-08-07 10:15:13 +00:00

aichrist commented

2024-08-27 16:43:31 +00:00

@shanduur any reasons why this driver should live in-tree? Most CSI drivers live outside of their storage backend codebase. It is likely easier to create a thin wrapper in go aside of garage than to do this in Rust.

@maximilien as a k8s user, I would expect this to be automatically installed by the helm chart referenced at https://garagehq.deuxfleurs.fr/documentation/cookbook/kubernetes/, with a helm var to disable if desired. I would have it enabled by default because we wouldn't be installing the helm chart if we weren't wanting to integrate garage with k8s.

Garage works well on k8s as-is, but there is a minor pain point in managing buckets/keys. It'd be great to be able to included these resources like Bucket or BucketAccess in our k8s configs and have object storage "just work."

> @shanduur any reasons why this driver should live in-tree? Most CSI drivers live outside of their storage backend codebase. It is likely easier to create a thin wrapper in go aside of garage than to do this in Rust. @maximilien as a k8s user, I would expect this to be automatically installed by the helm chart referenced at https://garagehq.deuxfleurs.fr/documentation/cookbook/kubernetes/, with a helm var to disable if desired. I would have it enabled by default because we wouldn't be installing the helm chart if we weren't wanting to integrate garage with k8s. Garage works well on k8s as-is, but there is a minor pain point in managing buckets/keys. It'd be great to be able to included these resources like `Bucket` or `BucketAccess` in our k8s configs and have object storage "just work."

No milestone

No project

No assignees

3 participants

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Deuxfleurs/garage#834

No description provided.