Commit graph

33 commits

Author SHA1 Message Date
Alex 80e4abb98d Merge pull request 'Changed all instances of 'key new' to 'key create' to make it the same as the bucket commands.' (#459) from jpds/garage:key-create-standardize into next
All checks were successful
continuous-integration/drone/push Build is passing
Reviewed-on: #459
2023-01-04 10:35:49 +00:00
Alex 570e5e5bbb
Merge branch 'main' into next
All checks were successful
continuous-integration/drone/push Build is passing
2023-01-04 11:34:43 +01:00
Jonathan Davies 8be862aa19 Changed all instances of 'key new' to 'key create' to make it consistent as bucket commands issued normally around the same time.
All checks were successful
continuous-integration/drone/pr Build is passing
2023-01-03 11:11:12 +00:00
Alex d7f90cabb0
Implement block retry-now and block purge
Some checks reported errors
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
continuous-integration/drone Build was killed
2022-12-13 15:02:42 +01:00
Alex 687660b27f
Implement block list-errors and block info
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2022-12-13 14:23:45 +01:00
Alex 9d82196945
cli: new worker info command
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-12-13 12:24:30 +01:00
Alex a51e8d94c6
cli: rename resync-n-workers into resync-worker-count
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2022-12-13 11:44:11 +01:00
Alex 73a4ca8b15
Use bytes as capacity units
Some checks failed
continuous-integration/drone/push Build is failing
continuous-integration/drone/pr Build is failing
2022-11-07 21:12:11 +01:00
Alex 28d7a49f63
Merge branch 'main' into optimal-layout
Some checks failed
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2022-11-07 12:20:59 +01:00
Alex 5b18fd8201
Add garage bucket cleanup-incomplete-uploads command
Some checks reported errors
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
continuous-integration/drone Build was killed
2022-11-04 11:55:59 +01:00
Mendes 4abab246f1 cargo fmt
Some checks failed
continuous-integration/drone/push Build is failing
continuous-integration/drone/pr Build is failing
2022-10-10 17:21:13 +02:00
Mendes a951b6c452 Added a CLI command to update the parameters for the layout computation (for now, only the zone redundancy)
Some checks failed
continuous-integration/drone/push Build is failing
continuous-integration/drone/pr Build is failing
2022-10-05 16:04:19 +02:00
Alex ceb1f0229a
Move version back into util
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-09-07 18:36:46 +02:00
Alex 06df301de5
Fix merge
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-09-07 18:16:01 +02:00
Alex 8adc654713
Merge branch 'main' into improve-deps 2022-09-07 18:13:27 +02:00
Alex db61f41030
Move GIT_VERSION injection later in build chain to reduce build times
Some checks failed
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is passing
2022-09-07 11:59:56 +02:00
Alex 5d4b937a00
Ability to have up to 4 concurrently working resync workers
Some checks failed
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2022-09-02 17:18:13 +02:00
Alex 943d76c583
Ability to dynamically set resync tranquility
All checks were successful
continuous-integration/drone/push Build is passing
2022-09-02 15:34:21 +02:00
Quentin 2c7bae935a
Configure structopt to report the right version
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/tag Build is passing
continuous-integration/drone Build is passing
continuous-integration/drone/push Build is passing
By default, structopt reports the value provided by
the env var CARGO_PKG_VERSION, feeded by Cargo when reading
Cargo.toml. However for Garage we use a versioning based on git,
so we often report a version that is behind the real version.
In this commit, we create garage_util::version::garage() that
reports the right version and configure all structopt subcommands
to call this function instead of using the env var.
2022-08-11 10:21:45 +02:00
Alex 4f38cadf6e Background task manager (#332)
All checks were successful
continuous-integration/drone/push Build is passing
- [x] New background worker trait
- [x] Adapt all current workers to use new API
- [x] Command to list currently running workers, and whether they are active, idle, or dead
- [x] Error reporting
- Optimizations
  - [x] Merkle updater: several items per iteration
  - [ ] Use `tokio::task::spawn_blocking` where appropriate so that CPU-intensive tasks don't block other things going on
- scrub:
  - [x] have only one worker with a channel to start/pause/cancel
  - [x] automatic scrub
  - [x] ability to view and change tranquility from CLI
  - [x] persistence of a few info
- [ ] Testing

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: #332
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-07-08 13:30:26 +02:00
Alex 77e3fd6db2 improve internal item counter mechanisms and implement bucket quotas (#326)
All checks were successful
continuous-integration/drone/push Build is passing
- [x] Refactoring of internal counting API
- [x] Repair procedure for counters (it's an offline procedure!!!)
- [x] New counter for objects in buckets
- [x] Add quotas to buckets struct
- [x] Add CLI to manage bucket quotas
- [x] Add admin API to manage bucket quotas
- [x] Apply quotas by adding checks on put operations
- [x] Proof-read

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: #326
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-06-15 20:20:28 +02:00
Alex 2814d41842
Allow garage layout assign to assign to several nodes at once 2022-03-24 15:27:13 +01:00
Alex 9eb211948e
Allow setting index document and error document on the CLI 2022-01-13 14:25:19 +01:00
Alex 9431090b1e
Implement key allow|deny --create-bucket
Some checks failed
continuous-integration/drone/push Build is failing
2022-01-05 15:12:59 +01:00
Alex 4d30e62db4
New buckets for 0.6.0: migration code and build files 2022-01-04 12:46:13 +01:00
Alex 0bbb6673e7
Model changes 2022-01-04 12:45:52 +01:00
Alex 53f71b3a57
Implement bucket alias and bucket unalias 2022-01-04 12:45:51 +01:00
Alex c94406f428
Improve how node roles are assigned in Garage
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
continuous-integration/drone Build is passing
- change the terminology: the network configuration becomes the role
  table, the configuration of a nodes becomes a node's role
- the modification of the role table takes place in two steps: first,
  changes are staged in a CRDT data structure. Then, once the user is
  happy with the changes, they can commit them all at once (or revert
  them).
- update documentation
- fix tests
- implement smarter partition assignation algorithm

This patch breaks the format of the network configuration: when
migrating, the cluster will be in a state where no roles are assigned.
All roles must be re-assigned and commited at once. This migration
should not pose an issue.
2021-11-16 16:05:53 +01:00
Alex 74a7a550eb
Safety: never voluntarily delete block in 10min interval after RC reaches zero 2021-11-08 15:47:47 +01:00
Alex 2090a6187f
Add tranquilizer mechanism to improve on token bucket mechanism
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2021-11-04 13:26:59 +01:00
Alex 6b47c294f5
Refactoring on repair commands
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2021-10-27 11:14:55 +02:00
Trinity Pointard 28c015d9ff
add cli parameter to verify local bloc integrity
All checks were successful
continuous-integration/drone/push Build is passing
reuse code for listing local blocks
add disk i/o speed limit on integrity check
2021-10-27 10:31:03 +02:00
Alex de4276202a
Improve CLI, adapt tests, update documentation 2021-10-25 14:21:48 +02:00