dc0b78cdb8
[block-ref-repair] Block refcount recalculation and repair
...
- We always recalculate the reference count of a block before deleting
it locally, to make sure that it is indeed zero.
- If we had to fetch a remote block but we were not able to get it,
check that refcount is indeed > 0.
- Repair procedure that checks everything
2024-03-19 16:20:22 +01:00
0038ca8a78
Merge branch 'main' into next-0.10
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/cron/debug Pipeline was successful
ci/woodpecker/cron/release/4 Pipeline was successful
ci/woodpecker/cron/release/3 Pipeline was successful
ci/woodpecker/cron/release/2 Pipeline was successful
ci/woodpecker/cron/release/1 Pipeline was successful
ci/woodpecker/cron/publish Pipeline was successful
2024-03-18 20:19:30 +01:00
a68c37555d
[db-snapshot] add garage meta snapshot cli operation
2024-03-15 13:51:31 +01:00
81191d2d92
Merge pull request 'Remove Sled' ( #767 ) from rm-sled into next-0.10
...
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/cron/release/3 Pipeline was successful
ci/woodpecker/cron/release/2 Pipeline was successful
ci/woodpecker/cron/debug Pipeline was successful
ci/woodpecker/cron/release/4 Pipeline was successful
ci/woodpecker/cron/release/1 Pipeline was successful
ci/woodpecker/cron/publish Pipeline was successful
Reviewed-on: #767
2024-03-12 10:45:57 +00:00
05c92204ec
[rm-sled] Remove counted_tree_hack
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2024-03-08 15:09:57 +01:00
f537f76681
[rm-migration] Remove migration path from Garage v0.5
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/deployment/debug Pipeline was successful
ci/woodpecker/deployment/release/3 Pipeline was successful
ci/woodpecker/deployment/release/2 Pipeline was successful
ci/woodpecker/deployment/release/4 Pipeline was successful
ci/woodpecker/deployment/release/1 Pipeline was successful
ci/woodpecker/deployment/publish Pipeline was successful
2024-03-08 13:24:47 +01:00
cf2af186fc
Merge branch 'main' into next-0.10
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2024-02-13 11:36:28 +01:00
82a29bf6e5
help, comments: make clear that full-length node ID = public key
...
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
Generally, avoid using the "public key" terminology
2024-01-16 14:04:11 +01:00
aa59059a91
layout cli: safer skip-dead-nodes command
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2023-12-07 11:56:14 +01:00
11e6fef93c
cli: add layout history and layout assume-sync commands
2023-11-27 16:22:25 +01:00
523d2ecb95
layout: use separate CRDT for staged layout changes
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2023-11-09 11:19:43 +01:00
d4932c31ea
Merge branch 'main' into next
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2023-10-10 13:57:21 +02:00
2d5d7a7031
Move convert_db command into main garage binary
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2023-10-10 12:13:15 +02:00
920dec393a
cli: more precise doc comment
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2023-10-04 10:44:42 +02:00
015ccb39aa
new layout: make zone_redundancy optionnal (if not set, is maximum)
continuous-integration/drone/push Build is failing
continuous-integration/drone/pr Build is failing
2023-09-18 11:59:08 +02:00
6b008b5bd3
block manager: add rebalance operation to rebalance multi-hdd setups
2023-09-07 13:44:11 +02:00
2e90e1c124
Merge branch 'main' into next
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build was killed
continuous-integration/drone Build is passing
2023-08-29 11:32:42 +02:00
b1ca1784a1
src/garage/cli: fix typo
2023-07-14 14:25:33 +03:00
7895f99d3a
admin and cli: hide secret keys unless asked
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2023-06-14 16:56:15 +02:00
7126f3e1d1
garage key import: add checks and --yes
CLI flag ( fix #278 )
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2023-06-13 15:56:48 +02:00
511e07ecd4
fix mpu counter (add missing workers) and report info at appropriate places
2023-06-09 16:23:37 +02:00
4ea53dc759
Add multipart upload repair
2023-06-09 16:23:37 +02:00
fa78d806e3
Merge branch 'main' into next
2023-04-25 12:34:26 +02:00
efb6b6e868
Disk space report
...
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
Report available disk space on nodes and calculate cluster-wide available space in `garage stats` (fix #479 )
2023-01-26 15:04:32 +01:00
Jonathan Davies
cb07e6145c
Changed all instances of assignation to assignment.
continuous-integration/drone/pr Build is passing
2023-01-05 11:09:25 +00:00
29dbcb8278
bg var operation on all nodes at once
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2023-01-04 13:25:57 +01:00
f3f27293df
Uniform framework for bg variable management
continuous-integration/drone/push Build is passing
2023-01-04 13:07:13 +01:00
80e4abb98d
Merge pull request 'Changed all instances of 'key new' to 'key create' to make it the same as the bucket commands.' ( #459 ) from jpds/garage:key-create-standardize into next
...
continuous-integration/drone/push Build is passing
Reviewed-on: #459
2023-01-04 10:35:49 +00:00
570e5e5bbb
Merge branch 'main' into next
continuous-integration/drone/push Build is passing
2023-01-04 11:34:43 +01:00
Jonathan Davies
8be862aa19
Changed all instances of 'key new' to 'key create' to make it consistent as bucket commands issued normally around the same time.
continuous-integration/drone/pr Build is passing
2023-01-03 11:11:12 +00:00
d7f90cabb0
Implement block retry-now
and block purge
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
continuous-integration/drone Build was killed
2022-12-13 15:02:42 +01:00
687660b27f
Implement block list-errors
and block info
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2022-12-13 14:23:45 +01:00
9d82196945
cli: new worker info command
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-12-13 12:24:30 +01:00
a51e8d94c6
cli: rename resync-n-workers into resync-worker-count
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2022-12-13 11:44:11 +01:00
73a4ca8b15
Use bytes as capacity units
continuous-integration/drone/push Build is failing
continuous-integration/drone/pr Build is failing
2022-11-07 21:12:11 +01:00
28d7a49f63
Merge branch 'main' into optimal-layout
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2022-11-07 12:20:59 +01:00
5b18fd8201
Add garage bucket cleanup-incomplete-uploads command
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
continuous-integration/drone Build was killed
2022-11-04 11:55:59 +01:00
4abab246f1
cargo fmt
continuous-integration/drone/push Build is failing
continuous-integration/drone/pr Build is failing
2022-10-10 17:21:13 +02:00
a951b6c452
Added a CLI command to update the parameters for the layout computation (for now, only the zone redundancy)
continuous-integration/drone/push Build is failing
continuous-integration/drone/pr Build is failing
2022-10-05 16:04:19 +02:00
ceb1f0229a
Move version back into util
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-09-07 18:36:46 +02:00
06df301de5
Fix merge
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-09-07 18:16:01 +02:00
8adc654713
Merge branch 'main' into improve-deps
2022-09-07 18:13:27 +02:00
db61f41030
Move GIT_VERSION injection later in build chain to reduce build times
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is passing
2022-09-07 11:59:56 +02:00
5d4b937a00
Ability to have up to 4 concurrently working resync workers
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2022-09-02 17:18:13 +02:00
943d76c583
Ability to dynamically set resync tranquility
continuous-integration/drone/push Build is passing
2022-09-02 15:34:21 +02:00
2c7bae935a
Configure structopt to report the right version
...
continuous-integration/drone/pr Build is passing
continuous-integration/drone/tag Build is passing
continuous-integration/drone Build is passing
continuous-integration/drone/push Build is passing
By default, structopt reports the value provided by
the env var CARGO_PKG_VERSION, feeded by Cargo when reading
Cargo.toml. However for Garage we use a versioning based on git,
so we often report a version that is behind the real version.
In this commit, we create garage_util::version::garage() that
reports the right version and configure all structopt subcommands
to call this function instead of using the env var.
2022-08-11 10:21:45 +02:00
4f38cadf6e
Background task manager ( #332 )
...
continuous-integration/drone/push Build is passing
- [x] New background worker trait
- [x] Adapt all current workers to use new API
- [x] Command to list currently running workers, and whether they are active, idle, or dead
- [x] Error reporting
- Optimizations
- [x] Merkle updater: several items per iteration
- [ ] Use `tokio::task::spawn_blocking` where appropriate so that CPU-intensive tasks don't block other things going on
- scrub:
- [x] have only one worker with a channel to start/pause/cancel
- [x] automatic scrub
- [x] ability to view and change tranquility from CLI
- [x] persistence of a few info
- [ ] Testing
Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: #332
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-07-08 13:30:26 +02:00
77e3fd6db2
improve internal item counter mechanisms and implement bucket quotas ( #326 )
...
continuous-integration/drone/push Build is passing
- [x] Refactoring of internal counting API
- [x] Repair procedure for counters (it's an offline procedure!!!)
- [x] New counter for objects in buckets
- [x] Add quotas to buckets struct
- [x] Add CLI to manage bucket quotas
- [x] Add admin API to manage bucket quotas
- [x] Apply quotas by adding checks on put operations
- [x] Proof-read
Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: #326
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-06-15 20:20:28 +02:00
2814d41842
Allow garage layout assign
to assign to several nodes at once
2022-03-24 15:27:13 +01:00
9eb211948e
Allow setting index document and error document on the CLI
2022-01-13 14:25:19 +01:00