Commit graph

2133 commits

Author SHA1 Message Date
7c8fc04b96 massively speed up compilation of garage_api_admin by not using async_trait
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2025-02-05 19:37:38 +01:00
f914db057a cli_v2: implement LaunchRepairOperation and remove old stuff
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
2025-02-05 15:36:47 +01:00
406b6da163 cli_v2: implement Get{Node,Cluster}Statistics
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
2025-02-05 15:06:10 +01:00
9f468b4439 cli_v2: implement CreateMetadataSnapshot
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
2025-02-05 14:22:10 +01:00
97be7b38fa admin api: reorder things
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2025-02-03 18:54:51 +01:00
6a1079c412 admin api: impl RequestHandler for MetricsRequest 2025-02-03 18:54:51 +01:00
b1629dd355 cli_v2: implement RetryBlockResync and PurgeBlocks 2025-02-03 18:54:51 +01:00
d405a9f839 cli_v2: implement ListBlockErrors and GetBlockInfo 2025-02-03 18:54:51 +01:00
7b9c047b11 cli_v2: add local_api_request with crazy type bound 2025-02-03 18:54:51 +01:00
10bbb26b30 cli_v2: implement ListWorkers and GetWorkerInfo 2025-02-03 18:54:51 +01:00
89ff9f5576 admin api: base infrastructure for local endpoints
admin api: rename EndpointHandler into RequestHandler to avoid confusion with RPC

wip: infrastructure for local api calls

admin api: fix things

admin api: first local endpoint to work with new scheme

admin api: implement SetWorkerVariable
2025-02-03 18:54:51 +01:00
bdaf55ab3f cli_v2: migrate cleanupincompleteuploads to Admin API
admin api: add CleanupIncompleteUploads spec
2025-02-03 18:54:51 +01:00
e96014ca60 Merge branch 'main' into next-v2
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
2025-02-03 18:53:17 +01:00
47e87c8739 Merge pull request 'upgrade Rust compiler and Cargo dependencies' (#951) from nix-crane into main
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/cron/debug Pipeline was successful
ci/woodpecker/cron/release/2 Pipeline was successful
ci/woodpecker/cron/release/1 Pipeline was successful
ci/woodpecker/cron/release/3 Pipeline was successful
ci/woodpecker/cron/release/4 Pipeline was successful
ci/woodpecker/cron/publish Pipeline was successful
Reviewed-on: #951
2025-02-03 17:49:00 +00:00
34599bff51 update all Cargo dependencies except AWS crates and their dependencies
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/deployment/debug Pipeline was successful
ci/woodpecker/deployment/release/3 Pipeline was successful
ci/woodpecker/deployment/release/2 Pipeline was successful
ci/woodpecker/deployment/release/1 Pipeline was successful
ci/woodpecker/deployment/release/4 Pipeline was successful
ci/woodpecker/deployment/publish Pipeline was successful
2025-02-03 17:46:54 +01:00
ec1a475923 build with rust 1.82.0 2025-02-03 17:46:48 +01:00
b9df2d1ad1 Merge pull request 'compile with crane' (#950) from nix-crane into main
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
Reviewed-on: #950
2025-02-03 15:54:54 +00:00
390a5d97fe nix, ci: build with Crane
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
This removes our dependency on cargo2nix, which was causing us some
issues. Whereas cargo2nix creates one Nix derivation per crate, Crane
uses only two derivations:

1. Build dependencies only
2. Build the final binary

This means that during the second step, no caching can be done. For
instance, if we do a change in garage_model, we need to recompile all of
the Garage crates including those that do not depend on garage_model.
On the upside, this allows all of the Garage crates to be built at once
using cargo build logic, which is optimized for high parallelism and
better pipelining between all of the steps of the build. All in all,
this makes most builds faster than cargo2nix.

A few other changes have been made to the build scripts and CI:

- Unit tests are now run within a Nix derivation. In fact, we have
  different derivations to run the tests using LMDB and Sqlite as
  metadata db engines.

- For debug builds, most CI steps now run in parallel (with the notable
  exception of the smoke test that runs after the build, which is
  inevitable).

- We no longer pass the GIT_VERSION argument when building debug builds
  and running the tests. This means that dev binaries and test
  binaries don't know the exact version of Garage they are from. That
  shouldn't be an issue in most cases.

- The not-dynamic.sh scripts has been fixed to fail if the file does not
  exist.
2025-02-03 16:39:50 +01:00
568c4954e9 Merge branch 'main' into next-v2 2025-02-01 19:58:36 +01:00
4dc2bc337f Merge pull request 'woodpecker: use parallel nix-build in debug builds' (#949) from nix-parallel into main
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/cron/debug Pipeline was successful
ci/woodpecker/cron/release/2 Pipeline was successful
ci/woodpecker/cron/release/1 Pipeline was successful
ci/woodpecker/cron/release/3 Pipeline was successful
ci/woodpecker/cron/release/4 Pipeline was successful
ci/woodpecker/cron/publish Pipeline was successful
Reviewed-on: #949
2025-02-01 18:58:15 +00:00
5dd2791981 woodpecker: use parallel nix-build in debug builds
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2025-02-01 19:48:01 +01:00
fe937c2901 Merge branch 'main' into next-v2
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2025-02-01 19:07:17 +01:00
d601f31186 Merge pull request 'split garage_api in garage_api_{common,s3,k2v,admin}' (#947) from split-garage-api into main
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
Reviewed-on: #947
2025-02-01 17:48:25 +00:00
e4de7bdfd5 fix ci for more test crates
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/deployment/debug Pipeline was successful
ci/woodpecker/deployment/release/3 Pipeline was successful
ci/woodpecker/deployment/release/2 Pipeline was successful
ci/woodpecker/deployment/release/1 Pipeline was successful
ci/woodpecker/deployment/release/4 Pipeline was successful
ci/woodpecker/deployment/publish Pipeline was successful
2025-01-31 19:21:36 +01:00
d18c5ad0ff fix tests
Some checks failed
ci/woodpecker/pr/debug Pipeline failed
ci/woodpecker/push/debug Pipeline failed
2025-01-31 19:12:51 +01:00
3d5e9a027e cargo defs: simplify and fix descriptions
Some checks failed
ci/woodpecker/pr/debug Pipeline failed
ci/woodpecker/push/debug Pipeline failed
2025-01-31 18:54:29 +01:00
f4ca7758b4 update cargo.nix
Some checks failed
ci/woodpecker/push/debug Pipeline failed
ci/woodpecker/pr/debug Pipeline failed
2025-01-31 18:48:07 +01:00
4563313f87 use cargo-shear to remove many unused dependencies between crates 2025-01-31 18:47:30 +01:00
afa28706e5 split s3/cors.rs into also common/cors.rs
Some checks failed
ci/woodpecker/push/debug Pipeline failed
2025-01-31 18:42:14 +01:00
84f1db91c4 fix things up 2025-01-31 18:34:57 +01:00
9fa20d45be wip: split garage_api into garage_api_{common,s3,k2v,admin} 2025-01-31 18:18:29 +01:00
9330fd79d3 Merge pull request 'table::insert_many: avoid failure with zero items (fix #915)' (#946) from fix-915 into main
Some checks failed
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/cron/debug Pipeline was successful
ci/woodpecker/cron/release/2 Pipeline failed
ci/woodpecker/cron/release/3 Pipeline failed
ci/woodpecker/cron/release/4 Pipeline failed
ci/woodpecker/cron/release/1 Pipeline failed
ci/woodpecker/cron/publish unknown status
Reviewed-on: #946
2025-01-31 13:10:54 +00:00
83f6928ff7 table::insert_many: avoid failure with zero items (fix #915)
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
2025-01-30 18:06:47 +01:00
3192088aac Merge pull request 'admin refactoring: refactor CLI to use Admin API requests (step 2)' (#943) from refactor-admin into next-v2
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
Reviewed-on: #943
2025-01-30 16:18:38 +00:00
5a89350b38 cli_v2: fix garage status
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
2025-01-30 16:45:59 +01:00
3caea5fc06 cli_v2: merge util.rs into layout.rs 2025-01-30 16:24:55 +01:00
ebc0e9319e cli_v2: error messages
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2025-01-30 16:17:35 +01:00
f8c6a8373d convert cli key operations to admin rpc
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2025-01-30 16:12:16 +01:00
076ce04fe5 fix garage status output
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2025-01-30 15:38:22 +01:00
f37d5d2b08 admin api: convert most bucket operations
Some checks failed
ci/woodpecker/push/debug Pipeline failed
ci/woodpecker/pr/debug Pipeline failed
2025-01-30 13:36:25 +01:00
819f4f0050 cli: migrate layout remove, apply, revert
Some checks failed
ci/woodpecker/push/debug Pipeline failed
ci/woodpecker/pr/debug Pipeline failed
2025-01-30 12:19:23 +01:00
69ddaafc60 wip: migrate garage status and garage layout assign
Some checks failed
ci/woodpecker/push/debug Pipeline failed
ci/woodpecker/pr/debug Pipeline failed
2025-01-30 12:07:12 +01:00
145130481e wip: proxy admin api requests through admin rpc, prepare new cli
Some checks failed
ci/woodpecker/push/debug Pipeline failed
2025-01-30 10:44:08 +01:00
6ed78abb5c Merge pull request 'admin API refactoring (step 1)' (#939) from refactor-admin into next-v2
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
Reviewed-on: #939
2025-01-29 20:42:56 +00:00
19454c1679 admin api: remove log message
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2025-01-29 19:47:37 +01:00
1c03941b19 admin api: fix panic on GetKeyInfo with no args
Some checks are pending
ci/woodpecker/push/debug Pipeline is running
ci/woodpecker/pr/debug Pipeline was successful
2025-01-29 19:26:16 +01:00
4f0b923c4f admin api: small fixes 2025-01-29 19:26:16 +01:00
420bbc162d admin api: clearer syntax for AddBucketAlias and RemoveBucketAlias 2025-01-29 19:26:16 +01:00
12ea4cda5f admin api: merge calls to manage global/local aliases 2025-01-29 19:26:16 +01:00
5fefbd94e9 admin api: rename allow/deny api calls in api v2 2025-01-29 19:26:16 +01:00