Commit graph

2135 commits

Author SHA1 Message Date
d46d449eb5 api: unify http precondition handling
Some checks failed
ci/woodpecker/pr/debug Pipeline failed
ci/woodpecker/push/debug Pipeline failed
2025-02-19 17:54:58 +01:00
1a8f74fc94 api: GetObject: implement if-match and if-unmodified-since 2025-02-19 17:26:29 +01:00
2191620af5 Merge pull request 'web: implement x-amz-website-redirect-location' (#966) from redirect-location-header into main
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
Reviewed-on: #966
2025-02-19 16:10:04 +00:00
bf27a3ec98 web: implement x-amz-website-redirect-location
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2025-02-19 17:04:10 +01:00
f64ec6e542 Merge pull request 'implement STREAMING-*-PAYLOAD-TRAILER' (#960) from fix-824 into main
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
Reviewed-on: #960
2025-02-19 09:59:32 +00:00
6d38907dac test: verify saved checksums in streaming putobject tests
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2025-02-18 22:02:04 +01:00
cfe8e8d45c api: PutObject: save trailer checksum in metadata
Some checks failed
ci/woodpecker/push/debug Pipeline failed
ci/woodpecker/pr/debug Pipeline was successful
2025-02-18 21:56:32 +01:00
f6e805e7db api: various fixes
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2025-02-18 21:47:53 +01:00
45e10e55f9 update aws-sdk-s3 in tests and fix wrong checksumming behavior in GetObject
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
2025-02-18 15:33:42 +01:00
730bfee753 api: validate trailing checksum + add test for unsigned-paylad-trailer 2025-02-18 15:33:42 +01:00
ccab0e4ae5 api: fix optional \n after trailer checksum header 2025-02-18 15:33:42 +01:00
abb60dcf7e api: remove content-encoding: aws-chunked for streaming payload 2025-02-18 15:33:42 +01:00
f8b0817ddc api: streaming signature: fix trailer parsing
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2025-02-18 12:00:41 +01:00
21c0dda16a api: refactor: move checksumming code around again
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2025-02-17 20:11:06 +01:00
658541d812 api: use checksumming in api_common::signature for put/putpart
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2025-02-17 19:54:25 +01:00
c5df820e2c api: start refactor of signature to calculate checksums earlier 2025-02-17 18:47:06 +01:00
a04d6cd5b8 api: streaming: parse unsigned streaming bodies and payload trailers
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
2025-02-17 16:23:24 +01:00
44a896f9b5 api: add logic to parse x-amz-content-sha256
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
2025-02-16 18:25:35 +01:00
cee7560fc1 api: refactor: move checksum algorithms to common 2025-02-16 17:25:55 +01:00
2f0c5ca220 signature: refactor: move constant defs to mod.rs 2025-02-16 16:34:18 +01:00
859b38b0d2 Merge pull request 'fix compilation warnings' (#959) from fixes into main
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/cron/debug Pipeline was successful
ci/woodpecker/cron/release/2 Pipeline was successful
ci/woodpecker/cron/release/1 Pipeline was successful
ci/woodpecker/cron/release/3 Pipeline was successful
ci/woodpecker/cron/release/4 Pipeline was successful
ci/woodpecker/cron/publish Pipeline was successful
Reviewed-on: #959
2025-02-14 17:32:30 +00:00
2729a71d9d fix warning in garage test
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2025-02-14 18:27:00 +01:00
c9d00f5f7b garage_api_s3: remove unused field in ListPartsQuery 2025-02-14 18:25:23 +01:00
89c944ebd6 Merge pull request 's3api: return Location in CompleteMultipartUpload (fix #852)' (#958) from fix-852 into main
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
Reviewed-on: #958
2025-02-14 17:16:58 +00:00
24470377c9 garage_model: fix warning about dead code 2025-02-14 18:12:14 +01:00
5b26545abf fix deprecated uses of chrono in lifecycle worker 2025-02-14 18:08:23 +01:00
9c7e3c7bde remove cargo build options in makefile to avoid mistakes 2025-02-14 18:06:07 +01:00
165f9316e2 s3api: return Location in CompleteMultipartUpload (fix #852)
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
NB. The location returned is not guaranteed to work in all cases.
This already fixes the parse issue in #852.
2025-02-14 18:05:07 +01:00
a94adf804f Merge pull request 'block manager: avoid deadlock in fix_block_location (fix #845)' (#957) from fix-845 into main
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
Reviewed-on: #957
2025-02-14 16:53:01 +00:00
e4c9a8cd53 block manager: avoid deadlock in fix_block_location (fix #845)
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
2025-02-14 17:41:50 +01:00
9312c6bbcb Merge pull request 'Store data blocks only on nodes in the latest cluster layout version (fix #815)' (#956) from fix-815 into main
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
Reviewed-on: #956
2025-02-14 15:53:16 +00:00
fdf4dad728 block resync: avoid saving blocks to draining nodes
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
2025-02-14 16:45:55 +01:00
6820b69f30 block manager: improve read strategy to find blocks faster 2025-02-14 16:45:55 +01:00
d0104b9f9b block manager: write blocks only to currently active layout version (fix #815)
avoid wastefully writing blocks to nodes that will discard them as soon
as the layout migration is finished
2025-02-14 16:45:55 +01:00
3fe8db9e52 Merge pull request 'web_server.rs: Added bucket domain to observability' (#608) from jpds/garage:domain-web-requests into main
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
Reviewed-on: #608
2025-02-14 14:26:08 +00:00
627a37fe9f Merge pull request 's3 api: parse x-id query parameter and warn of any inconsistency (fix #822)' (#954) from fix-822 into main
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
Reviewed-on: #954
2025-02-14 14:07:01 +00:00
2f55889835 add configuration option to enable/disable monitoring bucket in web metrics
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
2025-02-14 14:59:00 +01:00
Jonathan Davies
8b9cc5ca3f web_server.rs: Added bucket domain to observability. 2025-02-14 14:36:20 +01:00
a1533d2919 Merge pull request 'cli: return info of all nodes when doing garage stats -a (fix #814)' (#953) from fix-814 into main
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
Reviewed-on: #953
2025-02-14 13:31:42 +00:00
c1b39d9ba1 s3 api: parse x-id query parameter and warn of any inconsistency (fix #822)
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2025-02-14 14:30:58 +01:00
d84308c413 cli: return info of all nodes when doing garage stats -a (fix #814)
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
2025-02-14 14:11:41 +01:00
63f20bdeab Merge pull request 'db-snapshot: Add error handling to metadata snapshot creation' (#930) from handle_snapshot_errors into main
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
Reviewed-on: #930
Reviewed-by: Armael <armael@noreply.localhost>
2025-02-14 11:52:58 +00:00
Baptiste Jonglez
a2e134f036 db-snapshot: propagate any node snapshot error through RPC call
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
In particular, it means that "garage meta snapshot --all" will get an exit
code of 1 if any node fails to snapshot.

This makes sure that any external tool trying to snapshot nodes (e.g. from
cron) will be aware of the failure.

Fix #920
2025-02-07 00:29:43 +01:00
Baptiste Jonglez
06aa4b604f db-snapshot: Fix error reporting when using "garage meta snapshot --all"
Snapshot errors on remote nodes were not reported at all.

We now get proper error output such as:

    0fa0f35be69528ab  error: Internal error: DB error: LMDB: No space left on device (os error 28)
    88d92e2971d14bae  ok

Fix #920
2025-02-07 00:18:01 +01:00
d3226bfa91 Merge pull request 'remove uses of #[async_trait]' (#952) from remove-async-trait into main
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/cron/debug Pipeline was successful
ci/woodpecker/cron/release/2 Pipeline was successful
ci/woodpecker/cron/release/3 Pipeline was successful
ci/woodpecker/cron/release/1 Pipeline was successful
ci/woodpecker/cron/release/4 Pipeline was successful
ci/woodpecker/cron/publish Pipeline was successful
Reviewed-on: #952
2025-02-05 19:52:00 +00:00
af67626ab2 remove async_trait for TableRepair
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2025-02-05 20:45:07 +01:00
5475da8ea8 remove async_trait used in generic_server.rs
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
2025-02-05 20:31:34 +01:00
620dc58560 remove async_trait for traits declared in garage_net
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
2025-02-05 20:22:16 +01:00
47e87c8739 Merge pull request 'upgrade Rust compiler and Cargo dependencies' (#951) from nix-crane into main
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/cron/debug Pipeline was successful
ci/woodpecker/cron/release/2 Pipeline was successful
ci/woodpecker/cron/release/1 Pipeline was successful
ci/woodpecker/cron/release/3 Pipeline was successful
ci/woodpecker/cron/release/4 Pipeline was successful
ci/woodpecker/cron/publish Pipeline was successful
Reviewed-on: #951
2025-02-03 17:49:00 +00:00
34599bff51 update all Cargo dependencies except AWS crates and their dependencies
All checks were successful
ci/woodpecker/pr/debug Pipeline was successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/deployment/debug Pipeline was successful
ci/woodpecker/deployment/release/3 Pipeline was successful
ci/woodpecker/deployment/release/2 Pipeline was successful
ci/woodpecker/deployment/release/1 Pipeline was successful
ci/woodpecker/deployment/release/4 Pipeline was successful
ci/woodpecker/deployment/publish Pipeline was successful
2025-02-03 17:46:54 +01:00