Deuxfleurs/garagehq.deuxfleurs.fr
26
2
Fork 1
Code Issues 5 Pull requests 3 Projects Releases Wiki Activity

WIP: Add a roadmap draft #7

Draft
quentin wants to merge 1 commit from roadmap into master
pull from: roadmap
merge into: Deuxfleurs:master
Conversation 0 Commits 1 Files changed 1 +62
quentin commented 2022-04-07 09:39:35 +00:00
Owner
Copy link
No description provided.
quentin added 1 commit 2022-04-07 09:39:36 +00:00
Add a roadmap draft
All checks were successful
continuous-integration/drone/push Build is passing
Details
continuous-integration/drone/pr Build is passing
Details
a24e8b43d4
quentin added a new dependency 2022-04-07 09:39:51 +00:00
#6 Blog post inroducing Garage v0.7
lx reviewed 2022-04-12 12:58:43 +00:00
lx left a comment
Owner
Copy link

Added some notes on my perspective for the roadmap

Added some notes on my perspective for the roadmap
content/blog/2022-roadmap.md
@ -0,0 +16,4 @@
## Feature completeness
Most importantly, we still need to fix some corner cases on advanced S3 endpoints (eg. [#263](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues), [#248](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/248), [#204](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/204)). Based on community feedbacks, we might also consider implementing additional endpoints (eg. [#166](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/166)) or quotas (eg. [#71](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/71)) but we can't make any promise (sorry!).
lx commented 2022-04-12 12:35:14 +00:00
Owner
Copy link

#263 (anonymous access) -> this shouldn't be the first thing we evoke, for me it's low priority because we have the web endpoint that mostly serves the same purpose, however if there is a clear vision of what we want to achieve with this which currently can't be done, I'm interested

#248 (fix uploadpartcopy) -> we already have a patch for this, maybe not necessary to have it in the roadmap?

#166 (versionning) -> it looks to me that there is quite some demand for this, so it would be nice to do it, however it's a lot of work and we need to plan some time dedicated to this issue if we want to have it. Moreover it would imply a radical overhaul of the internal data structures of Garage.

#204 (correct multipart uploads) -> we will need to have this at some point; however it also requires changing the data model in a non-trivial way. If we decide to spend time on #166 we can do both at the same time, which would be simpler.

#71 (multi-tenancy) -> as Quentin said in the comments, we can start by a simpler accounting scheme already just to know how much storage is used by each bucket, which can be oportunistically done when we develop the index-counting mechanism for K2V. I don't really know the priority level for this but I'd guess quite low, I haven't heared of anybody who needs this now.

`#263` (anonymous access) -> this shouldn't be the first thing we evoke, for me it's low priority because we have the web endpoint that mostly serves the same purpose, however if there is a clear vision of what we want to achieve with this which currently can't be done, I'm interested `#248` (fix uploadpartcopy) -> we already have a patch for this, maybe not necessary to have it in the roadmap? `#166` (versionning) -> it looks to me that there is quite some demand for this, so it would be nice to do it, however it's a lot of work and we need to plan some time dedicated to this issue if we want to have it. Moreover it would imply a radical overhaul of the internal data structures of Garage. `#204` (correct multipart uploads) -> we will need to have this at some point; however it also requires changing the data model in a non-trivial way. If we decide to spend time on `#166` we can do both at the same time, which would be simpler. `#71` (multi-tenancy) -> as Quentin said in the comments, we can start by a simpler accounting scheme already just to know how much storage is used by each bucket, which can be oportunistically done when we develop the index-counting mechanism for K2V. I don't really know the priority level for this but I'd guess quite low, I haven't heared of anybody who needs this now.
content/blog/2022-roadmap.md
@ -0,0 +18,4 @@
Most importantly, we still need to fix some corner cases on advanced S3 endpoints (eg. [#263](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues), [#248](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/248), [#204](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/204)). Based on community feedbacks, we might also consider implementing additional endpoints (eg. [#166](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/166)) or quotas (eg. [#71](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/71)) but we can't make any promise (sorry!).
We also made a series of observation (the S3 API is not adapted to handle multiple small objects, many software require a database, we already have an internal database for metadata) which makes us believe that Garage could leverage its internal database system to provide a simple key-value store. We have already written [an API draft](https://p.adnab.me/code/#/2/code/view/eUNPbfoUrMbCY+CoMXaqed4jmWlmvWALHNDcfuM-O5o/embed/present/) for an API we named K2V. In the following months, we will then try to implement it and merge it if it makes sense. Spiritually, we would like it to be close to the original [Amazon Dynamo paper](https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf), or if you prefer approximative comparisons, K2V could be to Cassandra what sqlite is to PostgreSQL.
lx commented 2022-04-12 12:36:35 +00:00
Owner
Copy link

for the API draft please use the following URL: https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/k2v/doc/drafts/k2v-spec.md

Also I don't really like the comparison with sqlite, it feels wrong to me. I'd rather say we are to Cassandra what LMDB (or BerkeleyDB) is to Sqlite, but I guess nobody knows what LMDB or BerkeleyDB is.

for the API draft please use the following URL: <https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/k2v/doc/drafts/k2v-spec.md> Also I don't really like the comparison with sqlite, it feels wrong to me. I'd rather say we are to Cassandra what LMDB (or BerkeleyDB) is to Sqlite, but I guess nobody knows what LMDB or BerkeleyDB is.
content/blog/2022-roadmap.md
@ -0,0 +19,4 @@
Most importantly, we still need to fix some corner cases on advanced S3 endpoints (eg. [#263](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues), [#248](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/248), [#204](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/204)). Based on community feedbacks, we might also consider implementing additional endpoints (eg. [#166](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/166)) or quotas (eg. [#71](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/71)) but we can't make any promise (sorry!).
We also made a series of observation (the S3 API is not adapted to handle multiple small objects, many software require a database, we already have an internal database for metadata) which makes us believe that Garage could leverage its internal database system to provide a simple key-value store. We have already written [an API draft](https://p.adnab.me/code/#/2/code/view/eUNPbfoUrMbCY+CoMXaqed4jmWlmvWALHNDcfuM-O5o/embed/present/) for an API we named K2V. In the following months, we will then try to implement it and merge it if it makes sense. Spiritually, we would like it to be close to the original [Amazon Dynamo paper](https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf), or if you prefer approximative comparisons, K2V could be to Cassandra what sqlite is to PostgreSQL.
If you worry about feature bloat, be ensured that we do not plan to extend Garage beyond these points!
lx commented 2022-04-12 12:43:48 +00:00
Owner
Copy link

This looks a bit strange to me, how can we know in advance when we will stop or not? We don't really know who will use Garage in the future and what needs they will have.

This looks a bit strange to me, how can we know in advance when we will stop or not? We don't really know who will use Garage in the future and what needs they will have.
content/blog/2022-roadmap.md
@ -0,0 +27,4 @@
Garage has currently 2 API: S3 and Admin, with different *Consistency Models*. S3 API's consistency is aligned on [Amazon's new S3 Strong Consistency](https://aws.amazon.com/s3/consistency/) while Admin API's eventual consistency is not yet specified. We want to document first the Admin API's consistency, then we could make S3 consistency explanation more approachable.
The Admin API is also currently only exposed through our custom RPC endpoint. We would like to expose it through a REST endpoint ([#231](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/231)) to ease *Administration*. This REST API would make possible to build a web interface to manage Garage ([#232](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/232)).
lx commented 2022-04-12 12:53:51 +00:00
Owner
Copy link

#231 (REST admin endpoints) -> we will need this at Deuxfleurs to interconnect with Guichet so it's relatively high in the priority list. Also it's quite easy work which can probably be done without my intervention.

#232 (admin web UI) -> I don't think we should spend time developping a web UI just for Garage "in the general case", what we need for us is something specific that integrates with Guichet and we should focus on that

The following are missing in the list:

#207 monitoring of background tasks -> this would bring great quality-of-life improvements for Garage admins so I'd say it's pretty high in the list

#255 automatically scrub regularly -> for durability we need this; it would probably be better to develop it in conjunction with #207

`#231` (REST admin endpoints) -> we will need this at Deuxfleurs to interconnect with Guichet so it's relatively high in the priority list. Also it's quite easy work which can probably be done without my intervention. `#232` (admin web UI) -> I don't think we should spend time developping a web UI just for Garage "in the general case", what we need for us is something specific that integrates with Guichet and we should focus on that The following are missing in the list: `#207` monitoring of background tasks -> this would bring great quality-of-life improvements for Garage admins so I'd say it's pretty high in the list `#255` automatically scrub regularly -> for durability we need this; it would probably be better to develop it in conjunction with #207
content/blog/2022-roadmap.md
@ -0,0 +35,4 @@
<!--2) Explaining how Garage can take its place in the existing ecosystem, including among the other distributed storage systems, but also in term of uses cases and deployments (how does it perform at scale, with which hardware, for which application, etc.) 3) Make possible to manage Garage from a REST API, possibly write a web GUI to make administration easier, 4) help people understand the reliability and storage density they will have for a specific Garage deployment, if possible through a simulator, 5) we might consider adding a system of quota to protect a cluster from a misbehaving user. 6) Integration in ecosystems -->
## Correctness
lx commented 2022-04-12 10:10:18 +00:00
Owner
Copy link

this section could be called "correctness and performance"

this section could be called "correctness and performance"
content/blog/2022-roadmap.md
@ -0,0 +41,4 @@
But we still need to make sure that in practise our implementation is correct, and thus features these defined properties.
[Jepsen](https://jepsen.io/) is a well-known tool to test distributed system properties by simulating some system states and sequencing packets in a specific order.
We would like to learn it and apply it to Garage to better convince ourselves that we
lx commented 2022-04-12 12:54:44 +00:00
Owner
Copy link

Jepsen -> we don't have time, let's drop it for now

Jepsen -> we don't have time, let's drop it for now
content/blog/2022-roadmap.md
@ -0,0 +44,4 @@
We would like to learn it and apply it to Garage to better convince ourselves that we
Consider control theory, speak about tranquilizer ([#145](https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/145)).
lx commented 2022-04-12 12:55:34 +00:00
Owner
Copy link

Very low priority for me

Very low priority for me
content/blog/2022-roadmap.md
@ -0,0 +49,4 @@
Horizontal scaling made possible no leader also no erasure coding.
Track bottlenecks, instability and overload.
We also plan to deploy Garage on multiple clusters and do a large serie of benchmarks.
lx commented 2022-04-12 12:57:58 +00:00
Owner
Copy link

It looks here like we are entering into the "performance" domain. If so, we probably want to evoke a rework of the RPC stack to allow streaming RPCs (maybe based on QUIC but maybe not) which would allow to reduce the TTFB and general latency in many requests while also enabling us to increase the block size, bringing further improvements in performance. To me this is probably the highest priority item in the "performance" milestone.

It looks here like we are entering into the "performance" domain. If so, we probably want to evoke a rework of the RPC stack to allow streaming RPCs (maybe based on QUIC but maybe not) which would allow to reduce the TTFB and general latency in many requests while also enabling us to increase the block size, bringing further improvements in performance. To me this is probably the highest priority item in the "performance" milestone.
All checks were successful
continuous-integration/drone/push Build is passing
Details
continuous-integration/drone/pr Build is passing
Details
This pull request is marked as a work in progress.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin roadmap:roadmap
git checkout roadmap
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Depends on
#6 Blog post inroducing Garage v0.7
Deuxfleurs/garagehq.deuxfleurs.fr
Reference: Deuxfleurs/garagehq.deuxfleurs.fr#7
No description provided.
Delete branch "roadmap"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?