72 commits

Author	SHA1	Message	Date
Alex	fe2dc5d51c	Merge branch 'main' into next-0.10	2024-03-07 14:00:34 +01:00
Alex	eab2b81be2	[unicode-headers] allow utf-8 in headers + add test for object metadata	2024-03-07 13:42:01 +01:00
Alex	bbde9bc912	Merge branch 'main' into next-0.10	2024-03-04 15:56:10 +01:00
Alex	b6a91e549b	[test-presigned] Add API test for presigned requests	2024-03-04 13:02:07 +01:00
Alex	e9f759d4cb	[fix-presigned] presigned requests: allow x-amz-* query parameters to stand in for equivalent headers	2024-02-28 12:24:21 +01:00
Alex	a8cb8e8a8b	[fix-presigned] split presigned/normal signature verification	2024-02-28 12:24:13 +01:00
Alex	cf2af186fc	Merge branch 'main' into next-0.10	2024-02-13 11:36:28 +01:00
Alex	02e98e2d10	[header-override-650] implement header overriding in GetObject (fix #650)	2024-02-09 15:58:46 +01:00
Alex	10bc2ead60	[multi-char-delimiter-692] allow multi-character delimiters in List* (fix #692)	2024-02-09 14:15:29 +01:00
Alex	bcbd15da84	[dep-upgrade-202402] cargo clippy fixes	2024-02-08 23:29:57 +01:00
Alex	81ccd4586e	[dep-upgrade-202402] upgrade to http/hyper 1.x for tests	2024-02-05 19:57:35 +01:00
Alex	6e69a1fffc	[dep-upgrade-202402] prepare migration to http/hyper 1.0	2024-02-05 14:44:12 +01:00
Alex	6e4229e29c	[dep-upgrade-202402] update aws-sdk dependencies	2024-02-05 14:02:45 +01:00
Alex	46007bf01d	integration test: print stdout and stderr on subcommand crash	2023-11-15 12:56:52 +01:00
Alex	f97168f805	garage_db: refactor transactions and add on_commit mechanism	2023-09-21 15:35:31 +02:00
Alex	2e90e1c124	Merge branch 'main' into next	2023-08-29 11:32:42 +02:00
Alex	769b6fe054	fix test_website_check_domain	2023-08-28 12:40:28 +02:00
Alex	e66c78d6ea	integration test: move json_body to root of crate	2023-08-28 12:32:57 +02:00
Quentin	24e533f262	support {s3,web}.root_domains in /check endpoint	2023-08-08 11:05:42 +02:00
Jonathan Davies	a6da7e588f	tests/bucket.rs: Adjusted as previously used function is now private.	2023-06-26 18:03:43 +01:00
Jonathan Davies	7f8bf2d801	src/garage/tests: Updated types for aws-sdk-s3 bump.	2023-06-25 21:31:35 +01:00
Alex	90b2d43eb4	Merge branch 'main' into next	2023-06-13 17:14:11 +02:00
Alex	c14d3735e5	Add test for multipart uploads and fix part renumbering	2023-06-09 16:23:37 +02:00
Alex	8644376ac2	fix test; simplify code	2023-06-09 16:23:37 +02:00
Alex	1e466b11eb	Revert integration tests to using Sled as LMDB causes failures	2023-06-09 13:23:08 +02:00
Alex	e2ce5970c6	Add basic k2v_client integration tests	2023-05-22 10:45:06 +02:00
Alex	746b0090e4	k2v signature verification: double urlencoding (see comment in source code)	2023-05-18 11:18:06 +02:00
Alex	fa78d806e3	Merge branch 'main' into next	2023-04-25 12:34:26 +02:00
Quentin	70b5424b99	use one key per context to isolate tests	2023-03-13 15:06:05 +01:00
Quentin	2687fb7fa8	do not assume Garage boots in 2sec during tests	2023-03-13 15:06:05 +01:00
Jonathan Davies	9c354f0a8f	Improved bucket authorization response strings.	2023-01-29 20:34:41 +00:00
Jonathan Davies	004bb5b4f1	api_server.rs: Adapted to use query string per Caddy upstream change.	2023-01-29 20:34:37 +00:00
Alex	3113f6b5f2	more fixes	2023-01-26 17:14:17 +01:00
Alex	590a0a8450	Merge branch 'main' into k2v-watch-range-2	2023-01-26 16:46:40 +01:00
Jonathan Davies	36944f1839	Cargo.toml: Updated base64 from 0.13 to 0.21.	2023-01-26 11:13:07 +00:00
Jonathan Davies	4962b88f8b	tests/s3/website.rs: Added website hosting authorization check tests.	2023-01-13 09:39:02 +00:00
Alex	de1111076b	PollRange integration test	2023-01-11 10:04:41 +01:00
Alex	80e4abb98d	Merge pull request 'Changed all instances of 'key new' to 'key create' to make it the same as the bucket commands.' (#459) from jpds/garage:key-create-standardize into next ... Reviewed-on: #459	2023-01-04 10:35:49 +00:00
Alex	570e5e5bbb	Merge branch 'main' into next	2023-01-04 11:34:43 +01:00
Jonathan Davies	8be862aa19	Changed all instances of 'key new' to 'key create' to make it consistent as bucket commands issued normally around the same time.	2023-01-03 11:11:12 +00:00
Alex	0e61e3b6fb	Fix bucket creation tests to take permissions into account	2022-12-12 15:47:55 +01:00
Alex	fc2729cd81	Fix integration test	2022-11-08 15:19:46 +01:00
Tobias Krischer	7865003323	Use status code 204 No Content for empty responses	2022-10-17 10:55:26 +02:00
Alex	76f42a1a2b	Properly return HTTP 204 when deleting non-existent object (fix #227)	2022-09-14 17:07:55 +02:00
Alex	d9d199a6c9	Merge branch 'main' into lx-perf-improvements	2022-09-08 15:49:17 +02:00
Alex	1e92e9f782	Disable k2v tests when feature is disabled	2022-09-06 17:29:46 +02:00
Alex	ff4771c36a	cargo fmt	2022-07-29 12:24:48 +02:00
Alex	381eb9a5a1	Fix tests	2022-07-29 12:24:48 +02:00
Alex	aab34bfe54	add delays in k2v test_items_and_indices	2022-07-08 10:41:57 +02:00
Alex	b44d3fc796	Abstract database behind generic interface and implement alternative drivers (#322) ... - [x] Design interface - [x] Implement Sled backend - [x] Re-implement the SledCountedTree hack ~~on Sled backend~~ on all backends (i.e. over the abstraction) - [x] Convert Garage code to use generic interface - [x] Proof-read converted Garage code - [ ] Test everything well - [x] Implement sqlite backend - [x] Implement LMDB backend - [ ] (Implement Persy backend?) - [ ] (Implement other backends? (like RocksDB, ...)) - [x] Implement backend choice in config file and garage server module - [x] Add CLI for converting between DB formats - Exploit the new interface to put more things in transactions - [x] `.updated()` trigger on Garage tables Fix #284 **Bugs** - [x] When exporting sqlite, trees iterate empty?? - [x] LMDB doesn't work **Known issues for various back-ends** - Sled: - Eats all my RAM and also all my disk space - `.len()` has to traverse the whole table - Is actually quite slow on some operations - And is actually pretty bad code... - Sqlite: - Requires a lock to be taken on all operations. The lock is also taken when iterating on a table with `.iter()`, and the lock isn't released until the iterator is dropped. This means that we must be VERY carefull to not do anything else inside a `.iter()` loop or else we will have a deadlock! Most such cases have been eliminated from the Garage codebase, but there might still be some that remain. If your Garage-over-Sqlite seems to hang/freeze, this is the reason. - (adapter uses a bunch of unsafe code) - Heed (LMDB): - Not suited for 32-bit machines as it has to map the whole DB in memory. - (adpater uses a tiny bit of unsafe code) **My recommendation:** avoid 32-bit machines and use LMDB as much as possible. **Converting databases** is actually quite easy. For example from Sled to LMDB: ```bash cd src/db cargo run --features cli --bin convert -- -i path/to/garage/meta/db -a sled -o path/to/garage/meta/db.lmdb -b lmdb ``` Then, just add this to your `config.toml`: ```toml db_engine = "lmdb" ``` Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: #322 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>	2022-06-08 10:01:44 +02:00