garage/doc/book/src/intro.md

[ Download | Git repository | Matrix channel | Drone CI ]

This very website is hosted using Garage. In other words: the doc is the PoC!

The Garage Geo-Distributed Data Store

Garage is a lightweight geo-distributed data store that implements the Amazon S3 object storage protocole. It enables applications to store large blobs such as pictures, video, images, documents, etc., in a redundant multi-node setting. S3 is versatile enough to also be used to publish a static website.

Garage comes from the observation that despite the numerous existing implementation of object stores, many people have broken data management policies (backup/replication on a single site or none at all). To promote better data management policies, we focused on the following desirable properties:

• Self-contained & lightweight: works everywhere and integrates well in existing environments to target hyperconverged infrastructures.
• Highly resilient: highly resilient to network failures, network latency, disk failures, sysadmin failures.
• Simple: simple to understand, simple to operate, simple to debug.
• Internet enabled: made for multi-sites (eg. datacenters, offices, households, etc.) interconnected through regular Internet connections.

We also noted that the pursuit of some other goals are detrimental to our initial goals. The following has been identified as non-goals (if these points matter to you, you should not use Garage):

• Extreme performances: high performances constrain a lot the design and the infrastructure; we seek performances through minimalism only.
• Feature extensiveness: complete implementation of the S3 API or any other API to make Garage a drop-in replacement is not targeted as it could lead to decisions impacting our desirable properties.
• Storage optimizations: erasure coding or any other coding technique both increase the difficulty of placing data and synchronizing; we limit ourselves to duplication.
• POSIX/Filesystem compatibility: we do not aim at being POSIX compatible or to emulate any kind of filesystem. Indeed, in a distributed environment, such synchronizations are translated in network messages that impose severe constraints on the deployment.

Use Cases

Deuxfleurs: Garage is used by Deuxfleurs which is a non-profit hosting organization. Especially, it is used to host their main website, this documentation and some of its members' blogs. Deuxfleurs also uses Garage as their Matrix's media backend. Deuxfleurs also uses it in its continuous integration platform to store Drone's job logs and a Nix binary cache.

Are you using Garage? Open a pull request to add your organization here!

Comparison to existing software

MinIO: MinIO shares our Self-contained & lightweight goal but selected two of our non-goals: Storage optimizations through erasure coding and POSIX/Filesystem compatibility through strong consistency. However, by pursuing these two non-goals, MinIO do not reach our desirable properties. Firstly, it fails on the Simple property: due to the erasure coding, MinIO has severe limitations on how drives can be added or deleted from a cluster. Secondly, it fails on the Internet enabled property: due to its strong consistency, MinIO is latency sensitive. Furthermore, MinIO has no knowledge of "sites" and thus can not distribute data to minimize the failure of a given site.

Openstack Swift: OpenStack Swift at least fails on the Self-contained & lightweight goal. Starting it requires around 8GB of RAM, which is too much especially in an hyperconverged infrastructure. We also do not classify Swift as Simple.

Ceph: This review holds for the whole Ceph stack, including the RADOS paper, Ceph Object Storage module, the RADOS Gateway, etc. At its core, Ceph has been designed to provide POSIX/Filesystem compatibility which requires strong consistency, which in turn makes Ceph latency-sensitive and fails our Internet enabled goal. Due to its industry oriented design, Ceph is also far from being Simple to operate and from being Self-contained & lightweight which makes it hard to integrate it in an hyperconverged infrastructure. In a certain way, Ceph and MinIO are closer together than they are from Garage or OpenStack Swift.

More comparisons are available in our Related Work chapter.

Other Resources

This website is not the only source of information about Garage! We reference here other places on the Internet where you can learn more about Garage.

Rust API (docs.rs)

If you encounter a specific bug in Garage or plan to patch it, you may jump directly to the source code's documentation!

• garage_api - contains the S3 standard API endpoint
• garage_model - contains Garage's model built on the table abstraction
• garage_rpc - contains Garage's federation protocol
• garage_table - contains core Garage's CRDT datatypes
• garage_util - contains garage helpers
• garage_web - contains the S3 website endpoint

Talks

We love to talk and hear about Garage, that's why we keep a log here:

• (fr, 2021-11-13, video) Garage : Mille et une façons de stocker vos données and slides (html) - during RFID#1 event

• (en, 2021-04-28, pdf) Distributed object storage is centralised

• (fr, 2020-12-02, pdf) Garage : jouer dans la cour des grands quand on est un hébergeur associatif

Did you write or talk about Garage? Open a pull request to add a link here!

Community

If you want to discuss with us, you can join our Matrix channel at #garage:deuxfleurs.fr. Our code repository and issue tracker, which is the place where you should report bugs, is managed on Deuxfleurs' Gitea.

License

Garage's source code, is released under the AGPL v3 License. Please note that if you patch Garage and then use it to provide any service over a network, you must share your code!

Funding

The Deuxfleurs association has received a grant from NGI POINTER, to fund 3 people working on Garage full-time for a year: from October 2021 to September 2022.

This project has received funding from the European Union’s Horizon 2020 research and innovation programme within the framework of the NGI-POINTER Project funded under grant agreement N° 871528.