2107 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Alex	a94adf804f	Merge pull request 'block manager: avoid deadlock in fix_block_location (fix #845)' (#957) from fix-845 into main ... Reviewed-on: #957	2025-02-14 16:53:01 +00:00
Alex Auvolat	e4c9a8cd53	block manager: avoid deadlock in fix_block_location (fix #845)	2025-02-14 17:41:50 +01:00
Alex	9312c6bbcb	Merge pull request 'Store data blocks only on nodes in the latest cluster layout version (fix #815)' (#956) from fix-815 into main ... Reviewed-on: #956	2025-02-14 15:53:16 +00:00
Alex Auvolat	fdf4dad728	block resync: avoid saving blocks to draining nodes	2025-02-14 16:45:55 +01:00
Alex Auvolat	6820b69f30	block manager: improve read strategy to find blocks faster	2025-02-14 16:45:55 +01:00
Alex Auvolat	d0104b9f9b	block manager: write blocks only to currently active layout version (fix #815) ... avoid wastefully writing blocks to nodes that will discard them as soon as the layout migration is finished	2025-02-14 16:45:55 +01:00
Alex	3fe8db9e52	Merge pull request 'web_server.rs: Added bucket domain to observability' (#608) from jpds/garage:domain-web-requests into main ... Reviewed-on: #608	2025-02-14 14:26:08 +00:00
Alex	627a37fe9f	Merge pull request 's3 api: parse x-id query parameter and warn of any inconsistency (fix #822)' (#954) from fix-822 into main ... Reviewed-on: #954	2025-02-14 14:07:01 +00:00
Alex Auvolat	2f55889835	add configuration option to enable/disable monitoring bucket in web metrics	2025-02-14 14:59:00 +01:00
Jonathan Davies	8b9cc5ca3f	web_server.rs: Added bucket domain to observability.	2025-02-14 14:36:20 +01:00
Alex	a1533d2919	Merge pull request 'cli: return info of all nodes when doing garage stats -a (fix #814)' (#953) from fix-814 into main ... Reviewed-on: #953	2025-02-14 13:31:42 +00:00
Alex Auvolat	c1b39d9ba1	s3 api: parse x-id query parameter and warn of any inconsistency (fix #822)	2025-02-14 14:30:58 +01:00
Alex Auvolat	d84308c413	cli: return info of all nodes when doing garage stats -a (fix #814)	2025-02-14 14:11:41 +01:00
Alex	63f20bdeab	Merge pull request 'db-snapshot: Add error handling to metadata snapshot creation' (#930) from handle_snapshot_errors into main ... Reviewed-on: #930 Reviewed-by: Armael <armael@noreply.localhost>	2025-02-14 11:52:58 +00:00
Baptiste Jonglez	a2e134f036	db-snapshot: propagate any node snapshot error through RPC call ... In particular, it means that "garage meta snapshot --all" will get an exit code of 1 if any node fails to snapshot. This makes sure that any external tool trying to snapshot nodes (e.g. from cron) will be aware of the failure. Fix #920	2025-02-07 00:29:43 +01:00
Baptiste Jonglez	06aa4b604f	db-snapshot: Fix error reporting when using "garage meta snapshot --all" ... Snapshot errors on remote nodes were not reported at all. We now get proper error output such as: 0fa0f35be69528ab error: Internal error: DB error: LMDB: No space left on device (os error 28) 88d92e2971d14bae ok Fix #920	2025-02-07 00:18:01 +01:00
Alex	d3226bfa91	Merge pull request 'remove uses of #[async_trait]' (#952) from remove-async-trait into main ... Reviewed-on: #952	2025-02-05 19:52:00 +00:00
Alex Auvolat	af67626ab2	remove async_trait for TableRepair	2025-02-05 20:45:07 +01:00
Alex Auvolat	5475da8ea8	remove async_trait used in generic_server.rs	2025-02-05 20:31:34 +01:00
Alex Auvolat	620dc58560	remove async_trait for traits declared in garage_net	2025-02-05 20:22:16 +01:00
Alex	47e87c8739	Merge pull request 'upgrade Rust compiler and Cargo dependencies' (#951) from nix-crane into main ... Reviewed-on: #951	2025-02-03 17:49:00 +00:00
Alex Auvolat	34599bff51	update all Cargo dependencies except AWS crates and their dependencies	2025-02-03 17:46:54 +01:00
Alex Auvolat	ec1a475923	build with rust 1.82.0	2025-02-03 17:46:48 +01:00
Alex	b9df2d1ad1	Merge pull request 'compile with crane' (#950) from nix-crane into main ... Reviewed-on: #950	2025-02-03 15:54:54 +00:00
Alex Auvolat	390a5d97fe	nix, ci: build with Crane ... This removes our dependency on cargo2nix, which was causing us some issues. Whereas cargo2nix creates one Nix derivation per crate, Crane uses only two derivations: 1. Build dependencies only 2. Build the final binary This means that during the second step, no caching can be done. For instance, if we do a change in garage_model, we need to recompile all of the Garage crates including those that do not depend on garage_model. On the upside, this allows all of the Garage crates to be built at once using cargo build logic, which is optimized for high parallelism and better pipelining between all of the steps of the build. All in all, this makes most builds faster than cargo2nix. A few other changes have been made to the build scripts and CI: - Unit tests are now run within a Nix derivation. In fact, we have different derivations to run the tests using LMDB and Sqlite as metadata db engines. - For debug builds, most CI steps now run in parallel (with the notable exception of the smoke test that runs after the build, which is inevitable). - We no longer pass the GIT_VERSION argument when building debug builds and running the tests. This means that dev binaries and test binaries don't know the exact version of Garage they are from. That shouldn't be an issue in most cases. - The not-dynamic.sh scripts has been fixed to fail if the file does not exist.	2025-02-03 16:39:50 +01:00
Alex	4dc2bc337f	Merge pull request 'woodpecker: use parallel nix-build in debug builds' (#949) from nix-parallel into main ... Reviewed-on: #949	2025-02-01 18:58:15 +00:00
Alex Auvolat	5dd2791981	woodpecker: use parallel nix-build in debug builds	2025-02-01 19:48:01 +01:00
Alex	d601f31186	Merge pull request 'split garage_api in garage_api_{common,s3,k2v,admin}' (#947) from split-garage-api into main ... Reviewed-on: #947	2025-02-01 17:48:25 +00:00
Alex Auvolat	e4de7bdfd5	fix ci for more test crates	2025-01-31 19:21:36 +01:00
Alex Auvolat	d18c5ad0ff	fix tests	2025-01-31 19:12:51 +01:00
Alex Auvolat	3d5e9a027e	cargo defs: simplify and fix descriptions	2025-01-31 18:54:29 +01:00
Alex Auvolat	f4ca7758b4	update cargo.nix	2025-01-31 18:48:07 +01:00
Alex Auvolat	4563313f87	use cargo-shear to remove many unused dependencies between crates	2025-01-31 18:47:30 +01:00
Alex Auvolat	afa28706e5	split s3/cors.rs into also common/cors.rs	2025-01-31 18:42:14 +01:00
Alex Auvolat	84f1db91c4	fix things up	2025-01-31 18:34:57 +01:00
Alex Auvolat	9fa20d45be	wip: split garage_api into garage_api_{common,s3,k2v,admin}	2025-01-31 18:18:29 +01:00
Alex	9330fd79d3	Merge pull request 'table::insert_many: avoid failure with zero items (fix #915)' (#946) from fix-915 into main ... Reviewed-on: #946	2025-01-31 13:10:54 +00:00
Alex Auvolat	83f6928ff7	table::insert_many: avoid failure with zero items (fix #915)	2025-01-30 18:06:47 +01:00
Alex	ab71544499	Merge pull request 'api: better handling of helper errors to distinguish error codes' (#942) from fix-getkeyinfo-404 into main ... Reviewed-on: #942	2025-01-29 18:25:44 +00:00
Alex	991edbe02c	Merge pull request 'Update doc/book/connect/repositories.md' (#941) from yatesco/garage:main into main ... Reviewed-on: #941	2025-01-29 18:18:59 +00:00
Alex Auvolat	9f3c7c3720	api: better handling of helper errors to distinguish error codes	2025-01-29 19:14:34 +01:00
yatesco	bfde9152b8	Update doc/book/operations/multi-hdd.md ... trivial spelling mistake	2025-01-29 13:40:41 +00:00
yatesco	7bb042f0b7	Update doc/book/connect/repositories.md ... trivial spelling mistake	2025-01-29 13:34:35 +00:00
Alex	a1d081ee84	Merge pull request 's3 api: make x-amz-meta-* headers lowercase (fix #844)' (#938) from fix-844 into main ... Reviewed-on: #938	2025-01-27 19:32:19 +00:00
Alex Auvolat	e8fa89e834	s3 api: make x-amz-meta-* headers lowercase (fix #844)	2025-01-27 19:58:06 +01:00
Alex	beedc9fd11	Merge pull request 'snapshot: sqlite: use a subdirectory for consistency with LMDB' (#932) from baptiste/garage:snapshot_consistency_sqlite into main ... Reviewed-on: #932	2025-01-27 18:50:11 +00:00
Alex	d4e3e60920	Merge pull request 'update nix crate to 0.29 and libc to 0.2.169' (#931) from neuschaefer/garage:nix into main ... Reviewed-on: #931	2025-01-27 18:09:51 +00:00
Baptiste Jonglez	43402c9619	snapshot: sqlite: use a subdirectory for consistency with LMDB ... Currently, taking a snapshot of the metadata database with sqlite creates a sqlite file without extension with the following format: snapshots/2025-01-26T15:29:17Z This makes it hard to understand what kind of data this is, and is not consistent with LMDB: snapshots/2025-01-26T15:29:17Z/data.mdb With this change, we now get a directory with a single db.sqlite file: snapshots/2025-01-26T15:29:17Z/db.sqlite	2025-01-27 19:06:52 +01:00
Alex	efa6f3d85e	Merge pull request 'db-snapshot: allow to set directory where snapshots are stored' (#933) from baptiste/garage:configure_metadata_snapshots_dir into main ... Reviewed-on: #933	2025-01-27 18:04:05 +00:00
Alex Auvolat	74a1b49b13	Update Cargo.nix	2025-01-27 18:37:05 +01:00