Deuxfleurs/aerogramme.deuxfleurs.fr
17
0
Fork 0
Code Pull requests Activity

Add a roadmap draft

Browse source
This commit is contained in:
Quentin 2022-04-07 11:38:47 +02:00
parent 36f3a1783c
commit a24e8b43d4
1 changed files with 62 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

62
content/blog/2022-roadmap.md Normal file
View file

@ -0,0 +1,62 @@
+++
title="Our roadmap towards v1.0"
date=2022-04-10
+++
*a*
<!-- more -->
---
While we hope that Garage in its current state inspired you, we also understand that you may be curious about what will come next!
As per our road towards v1.0, we identified the remaining work and categorized it under one of these 3 domains: *Feature completeness*, *Understandability and manageability*, and *Correctness*.
## Feature completeness
Most importantly, we still need to fix some corner cases on advanced S3 endpoints (eg. [#263](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues), [#248](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/248), [#204](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/204)). Based on community feedbacks, we might also consider implementing additional endpoints (eg. [#166](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/166)) or quotas (eg. [#71](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/71)) but we can't make any promise (sorry!).
We also made a series of observation (the S3 API is not adapted to handle multiple small objects, many software require a database, we already have an internal database for metadata) which makes us believe that Garage could leverage its internal database system to provide a simple key-value store. We have already written [an API draft](https://p.adnab.me/code/#/2/code/view/eUNPbfoUrMbCY+CoMXaqed4jmWlmvWALHNDcfuM-O5o/embed/present/) for an API we named K2V. In the following months, we will then try to implement it and merge it if it makes sense. Spiritually, we would like it to be close to the original [Amazon Dynamo paper](https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf), or if you prefer approximative comparisons, K2V could be to Cassandra what sqlite is to PostgreSQL.
If you worry about feature bloat, be ensured that we do not plan to extend Garage beyond these points!
## Understandability and manageability
Based on community feedback (eg. [#228](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/228), [#221](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/221), [#217](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/217)), we have identified some points on which we want to dedicate work: *Consistency Model*, *Administration*, *Storage Model*, and *Ecosystem*.
Garage has currently 2 API: S3 and Admin, with different *Consistency Models*. S3 API's consistency is aligned on [Amazon's new S3 Strong Consistency](https://aws.amazon.com/s3/consistency/) while Admin API's eventual consistency is not yet specified. We want to document first the Admin API's consistency, then we could make S3 consistency explanation more approachable.
The Admin API is also currently only exposed through our custom RPC endpoint. We would like to expose it through a REST endpoint ([#231](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/231)) to ease *Administration*. This REST API would make possible to build a web interface to manage Garage ([#232](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/232)).
When people discover Garage, questions also arise around its *Storage Model*, eg. what reliability and density shoud they expect depending on their considered configuration. Ideally, we would like to write a calculator similar to [Minio's one](https://min.io/product/erasure-code-calculator) to help Garage operators understand the properties of their cluster.
Finally, Garage exists in an *Ecosystem*, consisting of multiple alternative software (eg. Ceph, Minio, SeaweedFS, IPFS Cluster), with different needs (research, customer services, archiving), made of various hardware (from a Raspberry Pi to a 64+ disk blade server). We want to situate Garage in its ecosystem while identifying, promoting and documenting some "Garage success story".
<!--2) Explaining how Garage can take its place in the existing ecosystem, including among the other distributed storage systems, but also in term of uses cases and deployments (how does it perform at scale, with which hardware, for which application, etc.) 3) Make possible to manage Garage from a REST API, possibly write a web GUI to make administration easier, 4) help people understand the reliability and storage density they will have for a specific Garage deployment, if possible through a simulator, 5) we might consider adding a system of quota to protect a cluster from a misbehaving user. 6) Integration in ecosystems -->
## Correctness
We know in theory that Garage's design works and horizontally scales.
But we still need to make sure that in practise our implementation is correct, and thus features these defined properties.
[Jepsen](https://jepsen.io/) is a well-known tool to test distributed system properties by simulating some system states and sequencing packets in a specific order.
We would like to learn it and apply it to Garage to better convince ourselves that we
Consider control theory, speak about tranquilizer ([#145](https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/145)).
Horizontal scaling made possible no leader also no erasure coding.
Track bottlenecks, instability and overload.
We also plan to deploy Garage on multiple clusters and do a large serie of benchmarks.
<!-- non goals: erasure coding, byzantine tolerance ; do we want quota? -->
## Conclusion
Please note that this roadmap is purely indicative, we are not committing to deliver these features.
Contributions are however welcome, so get in touch (on our Matrix channel) if you want to implement one of them.
We also don't know when v1.0 will be released, except "when it will be ready", but we would be happy if it could be by the end of 2022.
See you soon for the next release!