Deuxfleurs/garage
42
39
Fork 55
Code Issues 73 Pull requests 7 Releases 29 Wiki Activity
723 commits 24 branches 73 tags 55 MiB
Commit graph

28 commits

Author SHA1 Message Date
Alex 5768bf3622 First implementation of K2V (#293)
All checks were successful
continuous-integration/drone/push Build is passing
Details 
**Specification:**

View spec at [this URL](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/k2v/doc/drafts/k2v-spec.md)

- [x] Specify the structure of K2V triples
- [x] Specify the DVVS format used for causality detection
- [x] Specify the K2V index (just a counter of number of values per partition key)
- [x] Specify single-item endpoints: ReadItem, InsertItem, DeleteItem
- [x] Specify index endpoint: ReadIndex
- [x] Specify multi-item endpoints: InsertBatch, ReadBatch, DeleteBatch
- [x] Move to JSON objects instead of tuples
- [x] Specify endpoints for polling for updates on single values (PollItem)

**Implementation:**

- [x] Table for K2V items, causal contexts
- [x] Indexing mechanism and table for K2V index
- [x] Make API handlers a bit more generic
- [x] K2V API endpoint
- [x] K2V API router
- [x] ReadItem
- [x] InsertItem
- [x] DeleteItem
- [x] PollItem
- [x] ReadIndex
- [x] InsertBatch
- [x] ReadBatch
- [x] DeleteBatch

**Testing:**

- [x] Just a simple Python script that does some requests to check visually that things are going right (does not contain parsing of results or assertions on returned values)
- [x] Actual tests:
  - [x] Adapt testing framework
  - [x] Simple test with InsertItem + ReadItem
  - [x] Test with several Insert/Read/DeleteItem + ReadIndex
  - [x] Test all combinations of return formats for ReadItem
  - [x] Test with ReadBatch, InsertBatch, DeleteBatch
  - [x] Test with PollItem
  - [x] Test error codes
- [ ] Fix most broken stuff
  - [x] test PollItem broken randomly
  - [x] when invalid causality tokens are given, errors should be 4xx not 5xx

**Improvements:**

- [x] Descending range queries
  - [x] Specify
  - [x] Implement
  - [x] Add test
- [x] Batch updates to index counter
- [x] Put K2V behind `k2v` feature flag

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: #293
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
 2022-05-10 13:16:57 +02:00
Alex c1d9854d2c
Move block manager to separate module 2022-03-23 10:25:15 +01:00
Alex 9b2b531f4d
Make admin server optional
Some checks reported errors
continuous-integration/drone/push Build was killed
Details
continuous-integration/drone/pr Build was killed
Details
2022-03-14 10:54:25 +01:00
Alex dc8d0496cc
Refactoring: rename config files, make modifications less invasive 2022-03-14 10:53:51 +01:00
Alex 2a5609b292
Add metrics to API endpoint 2022-03-14 10:53:36 +01:00
Alex 818daa5c78
Refactor how durations are measured 2022-03-14 10:53:35 +01:00
Alex 8c2fb0c066
Add tracing integration with opentelemetry 2022-03-14 10:52:13 +01:00
Maximilien R 1e2cf26373
Implement basic metrics in table 2022-03-14 10:51:17 +01:00
mricher e349af13a7
Update dependencies and add admin module with metrics 
- Global dependencies updated in Cargo.lock
- New module created in src/admin to host:
  - the (future) admin REST API
  - the metric collection
- add configuration block

No metrics implemented yet
 2022-03-14 10:51:12 +01:00
Alex de4276202a
Improve CLI, adapt tests, update documentation 2021-10-25 14:21:48 +02:00
Alex 1b450c4b49
Improvements to CLI and various fixes for netapp version 
Discovery via consul, persist peer list to file
 2021-10-22 16:55:24 +02:00
Alex 4067797d01
First port of Garage to Netapp 2021-10-22 15:55:18 +02:00
Trinity Pointard 6644df6b96
fix clippy warnings on garage 2021-05-03 22:11:42 +02:00
Alex 575726358c
Tune Sled configuration 
- Make sled cache size and flush interval configurable
- Set less agressive default values:
  - cache size 128MB instead of 1GB
  - Flush interval 2 seconds instead of .5 seconds
 2021-05-03 17:27:43 +02:00
Trinity Pointard 1e3df189d0
document api crate 2021-04-27 16:37:10 +02:00
Trinity Pointard a2e1617d84
document web crate 2021-04-27 16:37:10 +02:00
Alex 9ced9f78dc
Improve bootstraping: do it regularly; persist peer list 2021-04-27 16:37:08 +02:00
Alex 6a8439fd13 Some improvements in background worker but we terminate late
All checks were successful
continuous-integration/drone/push Build is passing
Details
2021-03-15 23:14:12 +01:00
Alex 0cd5b2ae19 WIP migrate to tokio 1
All checks were successful
continuous-integration/drone/push Build is passing
Details
2021-03-15 22:36:41 +01:00
Alex 5ab33fddac Refactor CLI and prettify CLI outpu
All checks were successful
continuous-integration/drone/push Build is passing
Details
2021-03-12 18:12:31 +01:00
Alex 8d63738cb0 Checkpoint: add merkle tree in data table
All checks were successful
continuous-integration/drone/push Build is passing
Details
2021-03-11 13:47:21 +01:00
Alex 445912dc6a Remove migration paths from 0.1 branch
All checks were successful
continuous-integration/drone/push Build is passing
Details
2021-03-10 16:38:31 +01:00
Alex 20e6e9fa20 Update sled & try to debug deadlock (but its in sled...)
All checks were successful
continuous-integration/drone/push Build is passing
Details
2021-02-23 21:27:28 +01:00
Quentin 27795a390c Fix formatting 2020-11-10 09:59:52 +01:00
Quentin cea871d944 Skeleton to the new web API 2020-11-02 15:48:39 +01:00
Alex bec26a1312 Rename garage_core to garage_model 2020-07-07 13:59:22 +02:00
Alex fbe8fe81f2 Add automatic peer discovery from Consul 2020-06-30 18:33:14 +02:00
Alex d8f5e643bc Split code for modular compilation 2020-04-24 10:10:01 +00:00