2129 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Alex Auvolat	cfe8e8d45c	api: PutObject: save trailer checksum in metadata	2025-02-18 21:56:32 +01:00
Alex Auvolat	f6e805e7db	api: various fixes	2025-02-18 21:47:53 +01:00
Alex Auvolat	45e10e55f9	update aws-sdk-s3 in tests and fix wrong checksumming behavior in GetObject	2025-02-18 15:33:42 +01:00
Alex Auvolat	730bfee753	api: validate trailing checksum + add test for unsigned-paylad-trailer	2025-02-18 15:33:42 +01:00
Alex Auvolat	ccab0e4ae5	api: fix optional \n after trailer checksum header	2025-02-18 15:33:42 +01:00
Alex Auvolat	abb60dcf7e	api: remove content-encoding: aws-chunked for streaming payload	2025-02-18 15:33:42 +01:00
Alex Auvolat	f8b0817ddc	api: streaming signature: fix trailer parsing	2025-02-18 12:00:41 +01:00
Alex Auvolat	21c0dda16a	api: refactor: move checksumming code around again	2025-02-17 20:11:06 +01:00
Alex Auvolat	658541d812	api: use checksumming in api_common::signature for put/putpart	2025-02-17 19:54:25 +01:00
Alex Auvolat	c5df820e2c	api: start refactor of signature to calculate checksums earlier	2025-02-17 18:47:06 +01:00
Alex Auvolat	a04d6cd5b8	api: streaming: parse unsigned streaming bodies and payload trailers	2025-02-17 16:23:24 +01:00
Alex Auvolat	44a896f9b5	api: add logic to parse x-amz-content-sha256	2025-02-16 18:25:35 +01:00
Alex Auvolat	cee7560fc1	api: refactor: move checksum algorithms to common	2025-02-16 17:25:55 +01:00
Alex Auvolat	2f0c5ca220	signature: refactor: move constant defs to mod.rs	2025-02-16 16:34:18 +01:00
Alex	859b38b0d2	Merge pull request 'fix compilation warnings' (#959) from fixes into main ... Reviewed-on: #959	2025-02-14 17:32:30 +00:00
Alex Auvolat	2729a71d9d	fix warning in garage test	2025-02-14 18:27:00 +01:00
Alex Auvolat	c9d00f5f7b	garage_api_s3: remove unused field in ListPartsQuery	2025-02-14 18:25:23 +01:00
Alex	89c944ebd6	Merge pull request 's3api: return Location in CompleteMultipartUpload (fix #852)' (#958) from fix-852 into main ... Reviewed-on: #958	2025-02-14 17:16:58 +00:00
Alex Auvolat	24470377c9	garage_model: fix warning about dead code	2025-02-14 18:12:14 +01:00
Alex Auvolat	5b26545abf	fix deprecated uses of chrono in lifecycle worker	2025-02-14 18:08:23 +01:00
Alex Auvolat	9c7e3c7bde	remove cargo build options in makefile to avoid mistakes	2025-02-14 18:06:07 +01:00
Alex Auvolat	165f9316e2	s3api: return Location in CompleteMultipartUpload (fix #852) ... NB. The location returned is not guaranteed to work in all cases. This already fixes the parse issue in #852.	2025-02-14 18:05:07 +01:00
Alex	a94adf804f	Merge pull request 'block manager: avoid deadlock in fix_block_location (fix #845)' (#957) from fix-845 into main ... Reviewed-on: #957	2025-02-14 16:53:01 +00:00
Alex Auvolat	e4c9a8cd53	block manager: avoid deadlock in fix_block_location (fix #845)	2025-02-14 17:41:50 +01:00
Alex	9312c6bbcb	Merge pull request 'Store data blocks only on nodes in the latest cluster layout version (fix #815)' (#956) from fix-815 into main ... Reviewed-on: #956	2025-02-14 15:53:16 +00:00
Alex Auvolat	fdf4dad728	block resync: avoid saving blocks to draining nodes	2025-02-14 16:45:55 +01:00
Alex Auvolat	6820b69f30	block manager: improve read strategy to find blocks faster	2025-02-14 16:45:55 +01:00
Alex Auvolat	d0104b9f9b	block manager: write blocks only to currently active layout version (fix #815) ... avoid wastefully writing blocks to nodes that will discard them as soon as the layout migration is finished	2025-02-14 16:45:55 +01:00
Alex	3fe8db9e52	Merge pull request 'web_server.rs: Added bucket domain to observability' (#608) from jpds/garage:domain-web-requests into main ... Reviewed-on: #608	2025-02-14 14:26:08 +00:00
Alex	627a37fe9f	Merge pull request 's3 api: parse x-id query parameter and warn of any inconsistency (fix #822)' (#954) from fix-822 into main ... Reviewed-on: #954	2025-02-14 14:07:01 +00:00
Alex Auvolat	2f55889835	add configuration option to enable/disable monitoring bucket in web metrics	2025-02-14 14:59:00 +01:00
Jonathan Davies	8b9cc5ca3f	web_server.rs: Added bucket domain to observability.	2025-02-14 14:36:20 +01:00
Alex	a1533d2919	Merge pull request 'cli: return info of all nodes when doing garage stats -a (fix #814)' (#953) from fix-814 into main ... Reviewed-on: #953	2025-02-14 13:31:42 +00:00
Alex Auvolat	c1b39d9ba1	s3 api: parse x-id query parameter and warn of any inconsistency (fix #822)	2025-02-14 14:30:58 +01:00
Alex Auvolat	d84308c413	cli: return info of all nodes when doing garage stats -a (fix #814)	2025-02-14 14:11:41 +01:00
Alex	63f20bdeab	Merge pull request 'db-snapshot: Add error handling to metadata snapshot creation' (#930) from handle_snapshot_errors into main ... Reviewed-on: #930 Reviewed-by: Armael <armael@noreply.localhost>	2025-02-14 11:52:58 +00:00
Baptiste Jonglez	a2e134f036	db-snapshot: propagate any node snapshot error through RPC call ... In particular, it means that "garage meta snapshot --all" will get an exit code of 1 if any node fails to snapshot. This makes sure that any external tool trying to snapshot nodes (e.g. from cron) will be aware of the failure. Fix #920	2025-02-07 00:29:43 +01:00
Baptiste Jonglez	06aa4b604f	db-snapshot: Fix error reporting when using "garage meta snapshot --all" ... Snapshot errors on remote nodes were not reported at all. We now get proper error output such as: 0fa0f35be69528ab error: Internal error: DB error: LMDB: No space left on device (os error 28) 88d92e2971d14bae ok Fix #920	2025-02-07 00:18:01 +01:00
Alex	d3226bfa91	Merge pull request 'remove uses of #[async_trait]' (#952) from remove-async-trait into main ... Reviewed-on: #952	2025-02-05 19:52:00 +00:00
Alex Auvolat	af67626ab2	remove async_trait for TableRepair	2025-02-05 20:45:07 +01:00
Alex Auvolat	5475da8ea8	remove async_trait used in generic_server.rs	2025-02-05 20:31:34 +01:00
Alex Auvolat	620dc58560	remove async_trait for traits declared in garage_net	2025-02-05 20:22:16 +01:00
Alex	47e87c8739	Merge pull request 'upgrade Rust compiler and Cargo dependencies' (#951) from nix-crane into main ... Reviewed-on: #951	2025-02-03 17:49:00 +00:00
Alex Auvolat	34599bff51	update all Cargo dependencies except AWS crates and their dependencies	2025-02-03 17:46:54 +01:00
Alex Auvolat	ec1a475923	build with rust 1.82.0	2025-02-03 17:46:48 +01:00
Alex	b9df2d1ad1	Merge pull request 'compile with crane' (#950) from nix-crane into main ... Reviewed-on: #950	2025-02-03 15:54:54 +00:00
Alex Auvolat	390a5d97fe	nix, ci: build with Crane ... This removes our dependency on cargo2nix, which was causing us some issues. Whereas cargo2nix creates one Nix derivation per crate, Crane uses only two derivations: 1. Build dependencies only 2. Build the final binary This means that during the second step, no caching can be done. For instance, if we do a change in garage_model, we need to recompile all of the Garage crates including those that do not depend on garage_model. On the upside, this allows all of the Garage crates to be built at once using cargo build logic, which is optimized for high parallelism and better pipelining between all of the steps of the build. All in all, this makes most builds faster than cargo2nix. A few other changes have been made to the build scripts and CI: - Unit tests are now run within a Nix derivation. In fact, we have different derivations to run the tests using LMDB and Sqlite as metadata db engines. - For debug builds, most CI steps now run in parallel (with the notable exception of the smoke test that runs after the build, which is inevitable). - We no longer pass the GIT_VERSION argument when building debug builds and running the tests. This means that dev binaries and test binaries don't know the exact version of Garage they are from. That shouldn't be an issue in most cases. - The not-dynamic.sh scripts has been fixed to fail if the file does not exist.	2025-02-03 16:39:50 +01:00
Alex	4dc2bc337f	Merge pull request 'woodpecker: use parallel nix-build in debug builds' (#949) from nix-parallel into main ... Reviewed-on: #949	2025-02-01 18:58:15 +00:00
Alex Auvolat	5dd2791981	woodpecker: use parallel nix-build in debug builds	2025-02-01 19:48:01 +01:00
Alex	d601f31186	Merge pull request 'split garage_api in garage_api_{common,s3,k2v,admin}' (#947) from split-garage-api into main ... Reviewed-on: #947	2025-02-01 17:48:25 +00:00