Commit Graph

20 Commits

Author SHA1 Message Date
Alex 06df301de5
Fix merge
continuous-integration/drone/pr Build is passing Details
continuous-integration/drone/push Build is passing Details
2022-09-07 18:16:01 +02:00
Alex 8adc654713
Merge branch 'main' into improve-deps 2022-09-07 18:13:27 +02:00
Alex db61f41030
Move GIT_VERSION injection later in build chain to reduce build times
continuous-integration/drone/pr Build is failing Details
continuous-integration/drone/push Build is passing Details
2022-09-07 11:59:56 +02:00
Alex 5d4b937a00
Ability to have up to 4 concurrently working resync workers
continuous-integration/drone/pr Build is failing Details
continuous-integration/drone/push Build is failing Details
2022-09-02 17:18:13 +02:00
Alex 943d76c583
Ability to dynamically set resync tranquility
continuous-integration/drone/push Build is passing Details
2022-09-02 15:34:21 +02:00
Quentin 2c7bae935a
Configure structopt to report the right version
continuous-integration/drone/pr Build is passing Details
continuous-integration/drone/tag Build is passing Details
continuous-integration/drone Build is passing Details
continuous-integration/drone/push Build is passing Details
By default, structopt reports the value provided by
the env var CARGO_PKG_VERSION, feeded by Cargo when reading
Cargo.toml. However for Garage we use a versioning based on git,
so we often report a version that is behind the real version.
In this commit, we create garage_util::version::garage() that
reports the right version and configure all structopt subcommands
to call this function instead of using the env var.
2022-08-11 10:21:45 +02:00
Alex 4f38cadf6e Background task manager (#332)
continuous-integration/drone/push Build is passing Details
- [x] New background worker trait
- [x] Adapt all current workers to use new API
- [x] Command to list currently running workers, and whether they are active, idle, or dead
- [x] Error reporting
- Optimizations
  - [x] Merkle updater: several items per iteration
  - [ ] Use `tokio::task::spawn_blocking` where appropriate so that CPU-intensive tasks don't block other things going on
- scrub:
  - [x] have only one worker with a channel to start/pause/cancel
  - [x] automatic scrub
  - [x] ability to view and change tranquility from CLI
  - [x] persistence of a few info
- [ ] Testing

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: #332
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-07-08 13:30:26 +02:00
Alex 77e3fd6db2 improve internal item counter mechanisms and implement bucket quotas (#326)
continuous-integration/drone/push Build is passing Details
- [x] Refactoring of internal counting API
- [x] Repair procedure for counters (it's an offline procedure!!!)
- [x] New counter for objects in buckets
- [x] Add quotas to buckets struct
- [x] Add CLI to manage bucket quotas
- [x] Add admin API to manage bucket quotas
- [x] Apply quotas by adding checks on put operations
- [x] Proof-read

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: #326
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-06-15 20:20:28 +02:00
Alex 2814d41842
Allow `garage layout assign` to assign to several nodes at once 2022-03-24 15:27:13 +01:00
Alex 9eb211948e
Allow setting index document and error document on the CLI 2022-01-13 14:25:19 +01:00
Alex 9431090b1e
Implement key allow|deny --create-bucket
continuous-integration/drone/push Build is failing Details
2022-01-05 15:12:59 +01:00
Alex 4d30e62db4
New buckets for 0.6.0: migration code and build files 2022-01-04 12:46:13 +01:00
Alex 0bbb6673e7
Model changes 2022-01-04 12:45:52 +01:00
Alex 53f71b3a57
Implement bucket alias and bucket unalias 2022-01-04 12:45:51 +01:00
Alex c94406f428
Improve how node roles are assigned in Garage
continuous-integration/drone/pr Build is passing Details
continuous-integration/drone/tag Build is passing Details
continuous-integration/drone/push Build is passing Details
continuous-integration/drone Build is passing Details
- change the terminology: the network configuration becomes the role
  table, the configuration of a nodes becomes a node's role
- the modification of the role table takes place in two steps: first,
  changes are staged in a CRDT data structure. Then, once the user is
  happy with the changes, they can commit them all at once (or revert
  them).
- update documentation
- fix tests
- implement smarter partition assignation algorithm

This patch breaks the format of the network configuration: when
migrating, the cluster will be in a state where no roles are assigned.
All roles must be re-assigned and commited at once. This migration
should not pose an issue.
2021-11-16 16:05:53 +01:00
Alex 74a7a550eb
Safety: never voluntarily delete block in 10min interval after RC reaches zero 2021-11-08 15:47:47 +01:00
Alex 2090a6187f
Add tranquilizer mechanism to improve on token bucket mechanism
continuous-integration/drone/pr Build is passing Details
continuous-integration/drone/push Build is passing Details
2021-11-04 13:26:59 +01:00
Alex 6b47c294f5
Refactoring on repair commands
continuous-integration/drone/pr Build is passing Details
continuous-integration/drone/push Build is passing Details
2021-10-27 11:14:55 +02:00
Trinity Pointard 28c015d9ff
add cli parameter to verify local bloc integrity
continuous-integration/drone/push Build is passing Details
reuse code for listing local blocks
add disk i/o speed limit on integrity check
2021-10-27 10:31:03 +02:00
Alex de4276202a
Improve CLI, adapt tests, update documentation 2021-10-25 14:21:48 +02:00