garage/doc/book/src/intro.md

[ Download | Git repository | Matrix channel | Drone CI ]

Data resiliency for everyone

OLD

Garage is a lightweight geo-distributed data store that implements the Amazon S3 object storage protocole. It enables applications to store large blobs such as pictures, video, images, documents, etc., in a redundant multi-node setting. S3 is versatile enough to also be used to publish a static website.

Garage comes from the observation that despite the numerous existing implementation of object stores, many people have broken data management policies (backup/replication on a single site or none at all). To promote better data management policies, we focused on the following desirable properties:

Non-goals:

• Extreme performances: high performances constrain a lot the design and the infrastructure; we seek performances through minimalism only.
• Feature extensiveness: complete implementation of the S3 API or any other API to make Garage a drop-in replacement is not targeted as it could lead to decisions impacting our desirable properties.
• Storage optimizations: erasure coding or any other coding technique both increase the difficulty of placing data and synchronizing; we limit ourselves to duplication.
• POSIX/Filesystem compatibility: we do not aim at being POSIX compatible or to emulate any kind of filesystem. Indeed, in a distributed environment, such synchronizations are translated in network messages that impose severe constraints on the deployment.

Use-cases:

• Deuxfleurs: Garage is used by Deuxfleurs which is a non-profit hosting organization. Especially, it is used to host their main website, this documentation and some of its members' blogs. Deuxfleurs also uses Garage as their Matrix's media backend. Deuxfleurs also uses it in its continuous integration platform to store Drone's job logs and a Nix binary cache.

ENDOLD

Garage is an open-source distributed storage service you can self-host to fullfill many needs.

Garage implements the Amazon S3 API and thus is already compatible with many applications.

Garage provides data resiliency by replicating data 3x over distant servers.

Did you notice that this website is hosted and served by Garage?

Keeping requirements low

We worked hard to keep requirements as low as possible as we target the largest possible public.

• CPU: any x86_64 CPU from the last 10 years, ARMv7 or ARMv8.
• RAM: 1GB
• Disk Space: at least 16GB
• Network: 200ms or less, 50 Mbps or more

For the network, as we do not use consensus algorithms like Paxos or Raft, Garage is not as latency sensitive. Thanks to Rust and its zero-cost abstractions, we keep CPU and memory low.

Built on the shoulder of giants

• Dynamo: Amazon’s Highly Available Key-value Store by DeCandia et al.
• Conflict-Free Replicated Data Types by Shapiro et al.
• Maglev: A Fast and Reliable Software Network Load Balancer by Eisenbud et al.
• Merkle Search Trees: Efficient State-Based CRDTs in Open Networks by Auvolat and Taïani

Talks

We love to talk and hear about Garage, that's why we keep a log here:

• (fr, 2021-11-13, video) Garage : Mille et une façons de stocker vos données and slides (html) - during RFID#1 event

• (en, 2021-04-28, pdf) Distributed object storage is centralised

• (fr, 2020-12-02, pdf) Garage : jouer dans la cour des grands quand on est un hébergeur associatif

Did you write or talk about Garage? Open a pull request to add a link here!

Community

If you want to discuss with us, you can join our Matrix channel at #garage:deuxfleurs.fr. Our code repository and issue tracker, which is the place where you should report bugs, is managed on Deuxfleurs' Gitea.

License

Garage's source code, is released under the AGPL v3 License. Please note that if you patch Garage and then use it to provide any service over a network, you must share your code!

Sponsors and funding

The Deuxfleurs association has received a grant from NGI POINTER, to fund 3 people working on Garage full-time for a year: from October 2021 to September 2022.

This project has received funding from the European Union’s Horizon 2020 research and innovation programme within the framework of the NGI-POINTER Project funded under grant agreement N° 871528.