Update Related Work with a comparison table #221

New Issue

Open

opened 2022-02-04 11:39:43 +00:00 by quentin · 4 comments

quentin commented

2022-02-04 11:39:43 +00:00

Owner

We start to better understand the world of software defined storage so it may be time to have a more complete comparison of Garage against existing software.

I would see something of the following categories (may add some later):

user features (send people to the S3 matrix for more precise info)
- multipart uploads
- website hosting
- permission system
- versioning, retention, locking
- notifications
administration features
- supported architectures and systems
- security model
- flexible topology
- data reliability (scrub, repair)
- quotas
- telemetry, alerting
performances, optimizations
- minimal requirements
- latency sensitivity
- (writes) scaling
- crash tolerance
- erasure coding

We might also add some papers:

We start to better understand the world of software defined storage so it may be time to have a more complete comparison of Garage against existing software. I would see something of the following categories (may add some later): - user features (send people to the S3 matrix for more precise info) - multipart uploads - website hosting - permission system - versioning, retention, locking - notifications - administration features - supported architectures and systems - security model - flexible topology - data reliability (scrub, repair) - quotas - telemetry, alerting - performances, optimizations - minimal requirements - latency sensitivity - (writes) scaling - crash tolerance - erasure coding We might also add some papers: - https://assets.amazon.science/77/5e/4a7c238f4ce890efdc325df83263/using-lightweight-formal-methods-to-validate-a-key-value-storage-node-in-amazon-s3-2.pdf - https://papers.s3.fr-par.scw.cloud/hive.pdf

👍 2

quentin added the

Documentation

label 2022-02-04 11:39:43 +00:00

volth commented

2022-02-17 04:01:13 +00:00

Worth to compare to:

SeaweedFS
Apache Ozone

These look more like Garage than Minio and Ceph

Worth to compare to: * SeaweedFS * Apache Ozone These look more like Garage than Minio and Ceph

👍 1

baptiste commented

2022-04-09 11:38:06 +00:00

Owner

Also maybe Tahoe-LAFS: https://tahoe-lafs.readthedocs.io/en/latest/architecture.html

As far as I know it does not implement the S3 API, it's a custom API [1], but otherwise the goals are quite similar.

[1] https://tahoe-lafs.readthedocs.io/en/latest/frontends/webapi.html

Also maybe Tahoe-LAFS: https://tahoe-lafs.readthedocs.io/en/latest/architecture.html As far as I know it does not implement the S3 API, it's a custom API [1], but otherwise the goals are quite similar. [1] https://tahoe-lafs.readthedocs.io/en/latest/frontends/webapi.html

nora-puchreiner commented

2022-09-29 16:50:05 +00:00

Also, please compare with Ambry (https://github.com/linkedin/ambry)
It is the closest one to Garage (no erasure coding, just a copy in every DC, eventually consistent).

Also, please compare with Ambry (https://github.com/linkedin/ambry) It is the closest one to Garage (no erasure coding, just a copy in every DC, eventually consistent).

👍 1

quentin commented

2022-09-29 17:15:51 +00:00

Author

Owner

Thanks Nora, I just discovered the existence of Ambry.

Just to be precise, Garage does not perform a copy in every DC, it ensures, when possible, that you have a copy in 3 different DCs. If you have 5 DCs, these copies will be spread in such a way that you maximize your global storage. Garage has also stronger consistency properties on S3 endpoints, that are similar to Amazon's one. They refer to this property as strong read-after-write consistency: https://aws.amazon.com/fr/blogs/aws/amazon-s3-update-strong-read-after-write-consistency/

Thanks Nora, I just discovered the existence of Ambry. Just to be precise, Garage does not perform a copy in every DC, it ensures, when possible, that you have a copy in 3 different DCs. If you have 5 DCs, these copies will be spread in such a way that you maximize your global storage. Garage has also stronger consistency properties on S3 endpoints, that are similar to Amazon's one. They refer to this property as strong read-after-write consistency: https://aws.amazon.com/fr/blogs/aws/amazon-s3-update-strong-read-after-write-consistency/