Specify endpoints for polling for updates on single values (PollItem)
Implementation:
Table for K2V items, causal contexts
Indexing mechanism and table for K2V index
Make API handlers a bit more generic
K2V API endpoint
K2V API router
ReadItem
InsertItem
DeleteItem
PollItem
ReadIndex
InsertBatch
ReadBatch
DeleteBatch
Testing:
Just a simple Python script that does some requests to check visually that things are going right (does not contain parsing of results or assertions on returned values)
Actual tests:
Adapt testing framework
Simple test with InsertItem + ReadItem
Test with several Insert/Read/DeleteItem + ReadIndex
Test all combinations of return formats for ReadItem
Test with ReadBatch, InsertBatch, DeleteBatch
Test with PollItem
Test error codes
Fix most broken stuff
test PollItem broken randomly
when invalid causality tokens are given, errors should be 4xx not 5xx
Improvements:
Descending range queries
Specify
Implement
Add test
Batch updates to index counter
Put K2V behind experimental feature flag
**Specification:**
Making a Gitea PR as this is easier to edit and to comment than on cryptpad
View spec at [this URL](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/k2v/doc/drafts/k2v-spec.md)
- [x] Specify the structure of K2V triples
- [x] Specify the DVVS format used for causality detection
- [x] Specify the K2V index (just a counter of number of values per partition key)
- [x] Specify single-item endpoints: ReadItem, InsertItem, DeleteItem
- [x] Specify index endpoint: ReadIndex
- [x] Specify multi-item endpoints: InsertBatch, ReadBatch, DeleteBatch
- [x] Move to JSON objects instead of tuples
- [x] Specify endpoints for polling for updates on single values (PollItem)
**Implementation:**
- [x] Table for K2V items, causal contexts
- [x] Indexing mechanism and table for K2V index
- [x] Make API handlers a bit more generic
- [x] K2V API endpoint
- [x] K2V API router
- [x] ReadItem
- [x] InsertItem
- [x] DeleteItem
- [x] PollItem
- [x] ReadIndex
- [x] InsertBatch
- [x] ReadBatch
- [x] DeleteBatch
**Testing:**
- [x] Just a simple Python script that does some requests to check visually that things are going right (does not contain parsing of results or assertions on returned values)
- [x] Actual tests:
- [x] Adapt testing framework
- [x] Simple test with InsertItem + ReadItem
- [x] Test with several Insert/Read/DeleteItem + ReadIndex
- [x] Test all combinations of return formats for ReadItem
- [x] Test with ReadBatch, InsertBatch, DeleteBatch
- [x] Test with PollItem
- [x] Test error codes
- [ ] Fix most broken stuff
- [x] test PollItem broken randomly
- [x] when invalid causality tokens are given, errors should be 4xx not 5xx
**Improvements:**
- [x] Descending range queries
- [x] Specify
- [x] Implement
- [x] Add test
- [x] Batch updates to index counter
- [ ] Put K2V behind `experimental` feature flag
are triples and triplets the same thing? If they are different we should clarify how, and if they are the same, we should use only one word to name them.
are triples and triplets the same thing? If they are different we should clarify how, and if they are the same, we should use only one word to name them.
maybe singleItem should be true by default for delete operation, that would be safer against faulty API usage (it would probably require being the same for SEARCH, for the sake of coherence)
maybe `singleItem` should be true by default for delete operation, that would be safer against faulty API usage (it would probably require being the same for SEARCH, for the sake of coherence)
Specification:
Making a Gitea PR as this is easier to edit and to comment than on cryptpad
View spec at this URL
Implementation:
Testing:
Improvements:
experimental
feature flagWIP: K2V specificationto WIP: K2V 2 months ago26f44e5d85
to3d4d59e714
1 month agoquick review of k2v spec. Overall looks good, see comments for details
- K2V triplets exist separately from S3 objects. K2V triples don't exist for
the S3 API, and S3 objects don't exist for the K2V API.
- Values stored for triples have associated causality information, that enables
are triples and triplets the same thing? If they are different we should clarify how, and if they are the same, we should use only one word to name them.
- **Internal server errors on updates do not mean that the update isn't stored.**
K2V will return an internal server error when it cannot reach a quorum of nodes on
which to save an updated value. However the value may still be stored on just one
node, which whill then propagate it to other nodes asynchronously via anti-entropy.
which will
prefix: null,
start: null,
end: null,
singleItem: false,
maybe
singleItem
should be true by default for delete operation, that would be safer against faulty API usage (it would probably require being the same for SEARCH, for the sake of coherence)- a partition key (`pk`), an utf8 string that defines in what partition the triple is
stored; triples in different partitions cannot be listed together, they must
be the object of different ReadItem or ReadBatch queries
ReadItem reads a single triplet, so I don't think it's affected? Also, are other *Batch affected? I assume no, but this should probably be explicited
e4443d720d
toeaca736a4f
3 weeks agoWIP: K2Vto K2V 3 weeks ago5768bf3622
into main 3 weeks ago5768bf3622
.