Commit graph

31 commits

Author SHA1 Message Date
5768bf3622 First implementation of K2V (#293)
**Specification:**

View spec at [this URL](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/k2v/doc/drafts/k2v-spec.md)

- [x] Specify the structure of K2V triples
- [x] Specify the DVVS format used for causality detection
- [x] Specify the K2V index (just a counter of number of values per partition key)
- [x] Specify single-item endpoints: ReadItem, InsertItem, DeleteItem
- [x] Specify index endpoint: ReadIndex
- [x] Specify multi-item endpoints: InsertBatch, ReadBatch, DeleteBatch
- [x] Move to JSON objects instead of tuples
- [x] Specify endpoints for polling for updates on single values (PollItem)

**Implementation:**

- [x] Table for K2V items, causal contexts
- [x] Indexing mechanism and table for K2V index
- [x] Make API handlers a bit more generic
- [x] K2V API endpoint
- [x] K2V API router
- [x] ReadItem
- [x] InsertItem
- [x] DeleteItem
- [x] PollItem
- [x] ReadIndex
- [x] InsertBatch
- [x] ReadBatch
- [x] DeleteBatch

**Testing:**

- [x] Just a simple Python script that does some requests to check visually that things are going right (does not contain parsing of results or assertions on returned values)
- [x] Actual tests:
  - [x] Adapt testing framework
  - [x] Simple test with InsertItem + ReadItem
  - [x] Test with several Insert/Read/DeleteItem + ReadIndex
  - [x] Test all combinations of return formats for ReadItem
  - [x] Test with ReadBatch, InsertBatch, DeleteBatch
  - [x] Test with PollItem
  - [x] Test error codes
- [ ] Fix most broken stuff
  - [x] test PollItem broken randomly
  - [x] when invalid causality tokens are given, errors should be 4xx not 5xx

**Improvements:**

- [x] Descending range queries
  - [x] Specify
  - [x] Implement
  - [x] Add test
- [x] Batch updates to index counter
- [x] Put K2V behind `k2v` feature flag

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: Deuxfleurs/garage#293
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-05-10 13:16:57 +02:00
acdf893362
Fix partnumber 2022-01-25 12:25:23 +01:00
338b1b83ee
Implement part_number for GetObject 2022-01-24 21:04:42 +01:00
6dab836f3a
Multipart improvements
- support part_number for HeadObject
- add checks in complete_multipart_upload
2022-01-24 21:04:40 +01:00
168a90dfb5
Fix some error codes 2022-01-05 17:07:36 +01:00
5b1117e582
New model for buckets 2022-01-04 12:45:46 +01:00
7f26ed55cd Improved handling of HTTP ranges
- correct HTTP code when range syntax is invalid (fix #140)
- when multiple ranges are given, simply ignore and send whole file

Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com>
Reviewed-on: Deuxfleurs/garage#157
Reviewed-by: Alex <alex@adnab.me>
Co-authored-by: trinity-1686a <trinity.pointard@gmail.com>
Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
2021-11-29 11:52:42 +01:00
ada7899b24
Fix clippy lints (fix #121) 2021-10-26 10:20:05 +02:00
84856e84e5
fix clippy warnings on api 2021-05-03 22:11:41 +02:00
1e3df189d0
document api crate 2021-04-27 16:37:10 +02:00
6edbc65847 Add trinity's comment in the code 2021-03-18 19:46:43 +01:00
b4c903371c add support for caching headers 2021-03-18 15:46:33 +01:00
f319a7d374 Refactor model stuff, including cleaner CRDTs 2021-03-10 16:21:56 +01:00
02d512f3fd Fix #28, extra headers being ignored (because of profound stupidity) 2021-02-19 12:38:22 +01:00
4a5bbbb810 Propose ETag fix 2020-12-05 19:23:46 +01:00
76b489f3d3 Reformulate patch 2020-12-05 19:20:07 +01:00
bd7e3d1bd1 Fix Content-Length 2020-12-05 18:57:22 +01:00
9f46fb699a Content-range fix 2020-12-05 16:37:59 +01:00
5a5592c176 Replace with option syntaxic sugar 2020-11-11 16:12:42 +01:00
a50fa70d45 Refactor error management in API part 2020-11-08 15:05:28 +01:00
1c70552f95 Validate content MD5 and SHA256 sums for PutObject and UploadPart 2020-07-15 15:31:13 +02:00
9305e5e87f More headers taken into account 2020-07-09 17:04:43 +02:00
44dba0e53c cargo fmt 2020-07-08 17:34:37 +02:00
64a6eda0d2 Migrate S3 api to use new model 2020-07-08 17:33:24 +02:00
86fb7bbba5 Apply cargo fmt; add trace output when request signature is bad 2020-07-08 13:33:02 +02:00
f22ecb60a8 Update to Hyper 0.13.6 that accepts non-Sync streams in wrap_stream.
Simplifies code and makes it possible to publish on crates.io
2020-07-07 17:15:53 +02:00
bec26a1312 Rename garage_core to garage_model 2020-07-07 13:59:22 +02:00
b46a7788d1 Implement HTTP ranges in get 2020-05-04 13:09:23 +00:00
81ecc4999e Implement multipart uploads 2020-04-26 20:39:32 +00:00
9cb870f950 Prepare for multipart uploads 2020-04-26 18:55:13 +00:00
91b2d1fcc1 Some basic S3 functionnality 2020-04-24 18:47:11 +00:00