407 commits

Author	SHA1	Message	Date
Alex	aa7eadc799	Merge pull request 'New layout: fixes and UX improvements' (#634) from new-layout-ux into next ... Reviewed-on: #634	2023-09-27 09:04:32 +00:00
Alex	f97168f805	garage_db: refactor transactions and add on_commit mechanism	2023-09-21 15:35:31 +02:00
Alex	749b4865d0	new layout: improve display and fix comments	2023-09-18 12:07:45 +02:00
Alex	015ccb39aa	new layout: make zone_redundancy optionnal (if not set, is maximum)	2023-09-18 11:59:08 +02:00
Alex	2e229d4430	new layout: improve output display	2023-09-12 17:24:51 +02:00
Alex	aa79810596	Fix error when none of S3/K2V/WEB/ADMIN server is started (fix #613)	2023-09-12 14:35:19 +02:00
Alex	ad6b1cc0be	Merge branch 'main' into next	2023-09-11 13:14:18 +02:00
Alex	6b008b5bd3	block manager: add rebalance operation to rebalance multi-hdd setups	2023-09-07 13:44:11 +02:00
Alex	51eac97260	update version to 0.8.4	2023-09-05 23:28:12 +02:00
Alex	2e90e1c124	Merge branch 'main' into next	2023-08-29 11:32:42 +02:00
Alex	cece1be1bb	bump version to 0.8.3	2023-08-28 13:17:26 +02:00
Alex	769b6fe054	fix test_website_check_domain	2023-08-28 12:40:28 +02:00
Alex	e66c78d6ea	integration test: move json_body to root of crate	2023-08-28 12:32:57 +02:00
Jonathan Davies	5185701aa8	cargo: Updated: ... * addr2line v0.19.0 -> v0.20.0 * async-compression v0.4.0 -> v0.4.1 * clap v4.3.8 -> v4.3.19 * hyper v0.14.26 -> v0.14.27 * ipnet v2.7.2 -> v2.8.0 * rmp v0.8.11 -> v0.8.12 * serde v1.0.164 -> v1.0.188 * tokio v1.29.0 -> v1.31.0 * zstd v0.12.3+zstd.1.5.2 -> v0.12.4 * Others in `cargo update`	2023-08-28 09:30:27 +00:00
Quentin	24e533f262	support {s3,web}.root_domains in /check endpoint	2023-08-08 11:05:42 +02:00
Florian Klink	b1ca1784a1	src/garage/cli: fix typo	2023-07-14 14:25:33 +03:00
Jonathan Davies	a6da7e588f	tests/bucket.rs: Adjusted as previously used function is now private.	2023-06-26 18:03:43 +01:00
Jonathan Davies	7f8bf2d801	src/garage/tests: Updated types for aws-sdk-s3 bump.	2023-06-25 21:31:35 +01:00
Jonathan Davies	4297233d3e	garage/Cargo.toml: Updated aws-sdk-s3 to 0.28, added aws-config.	2023-06-25 21:17:15 +01:00
Alex	8ef42c9609	admin docs: reformatting, key admin: add check	2023-06-14 17:19:25 +02:00
Alex	7895f99d3a	admin and cli: hide secret keys unless asked	2023-06-14 16:56:15 +02:00
Alex	90b2d43eb4	Merge branch 'main' into next	2023-06-13 17:14:11 +02:00
Alex	eb9cecf05c	Split garage/admin.rs into smaller files	2023-06-13 16:46:28 +02:00
Alex	802ed75721	move admin.rs to admin/mod.rs, before splitting	2023-06-13 16:42:14 +02:00
Alex	7126f3e1d1	garage key import: add checks and --yes CLI flag (fix #278)	2023-06-13 15:56:48 +02:00
Alex	1ea4937c8b	fix timestamps wrapping around in garage block list-errors (fix #584)	2023-06-12 20:07:33 +02:00
Alex	c14d3735e5	Add test for multipart uploads and fix part renumbering	2023-06-09 16:23:37 +02:00
Alex	412ab77b08	comments and clippy lint fixes	2023-06-09 16:23:37 +02:00
Alex	511e07ecd4	fix mpu counter (add missing workers) and report info at appropriate places	2023-06-09 16:23:37 +02:00
Alex	4ea53dc759	Add multipart upload repair	2023-06-09 16:23:37 +02:00
Alex	058518c22b	refactor repair workers with a trait	2023-06-09 16:23:37 +02:00
Alex	8644376ac2	fix test; simplify code	2023-06-09 16:23:37 +02:00
Alex	75a0e01372	fix online repair	2023-06-09 16:23:37 +02:00
Alex	bb176ebcb8	cargo fmt	2023-06-09 16:23:37 +02:00
Alex	87be8eeb93	updaet block admin for new multipartupload models	2023-06-09 16:23:37 +02:00
Alex	1e466b11eb	Revert integration tests to using Sled as LMDB causes failures	2023-06-09 13:23:08 +02:00
Alex	8a74e1c2bd	Split garage/admin.rs into smaller files	2023-06-06 15:39:15 +02:00
Alex	e2ce5970c6	Add basic k2v_client integration tests	2023-05-22 10:45:06 +02:00
Alex	746b0090e4	k2v signature verification: double urlencoding (see comment in source code)	2023-05-18 11:18:06 +02:00
Alex	19639705e6	Mark sled as deprecated, make lmdb default, and improve sqlite and lmdb defaults	2023-05-17 14:30:53 +02:00
Alex	a1cec2cd60	Split format_table into separate crate and reduce k2v-client dependencies	2023-05-17 13:01:37 +02:00
Alex	351d734e6c	Merge branch 'main' into next	2023-05-09 12:40:08 +02:00
Alex	9e0a9c1c15	move git-version dependency to main crate to reduce rebuilds	2023-05-09 11:35:32 +02:00
Jakub Jirutka	d2deee0b8b	Declare garage crates using workspace.dependencies ... This will allow to really disable "sled" feature without declaring `default-features = false` in every Cargo.toml where garage_db and garage_model is used. See https://doc.rust-lang.org/cargo/reference/workspaces.html#the-dependencies-table	2023-05-09 08:46:15 +00:00
Jakub Jirutka	833cf082da	Remove unnecessary/unused "timeago" features ... To decrease dependency bloat and binary size.	2023-05-07 01:03:54 +02:00
Alex	fa78d806e3	Merge branch 'main' into next	2023-04-25 12:34:26 +02:00
Alex	0a1ddcf630	Prepare for v0.8.2	2023-03-13 18:46:31 +01:00
Alex	d6ffa57f40	Merge pull request 'Increase Garage tests robustness' (#526) from tests/increase-robustness into main ... Reviewed-on: #526 Reviewed-by: Alex <alex@adnab.me> Reviewed-by: trinity-1686a <trinity.pointard@gmail.com>	2023-03-13 17:26:21 +00:00
Quentin	70b5424b99	use one key per context to isolate tests	2023-03-13 15:06:05 +01:00
Quentin	2687fb7fa8	do not assume Garage boots in 2sec during tests	2023-03-13 15:06:05 +01:00
Jonathan Davies	4421378023	garage/admin.rs: Display Rust version in stats output.	2023-03-10 14:46:54 +00:00
Alex	80e2326998	fixes for pr 499	2023-02-06 12:23:55 +01:00
Alex	656b8d42de	secrets can be passed directly in config, as file, or as env	2023-02-03 15:27:39 +01:00
Jonathan Davies	9c354f0a8f	Improved bucket authorization response strings.	2023-01-29 20:34:41 +00:00
Jonathan Davies	004bb5b4f1	api_server.rs: Adapted to use query string per Caddy upstream change.	2023-01-29 20:34:37 +00:00
Alex	d6af95d205	fix cli display bug	2023-01-26 17:50:50 +01:00
Alex	3113f6b5f2	more fixes	2023-01-26 17:14:17 +01:00
Alex	1dff62564f	fix clippy	2023-01-26 17:05:31 +01:00
Alex	590a0a8450	Merge branch 'main' into k2v-watch-range-2	2023-01-26 16:46:40 +01:00
Alex	94d559ae00	Merge branch 'main' into report-disk-usage	2023-01-26 16:20:41 +01:00
Alex	efb6b6e868	Disk space report ... Report available disk space on nodes and calculate cluster-wide available space in `garage stats` (fix #479)	2023-01-26 15:04:32 +01:00
Jonathan Davies	20c1cdf662	Cargo.toml: Loosen tracing dependency to just 0.1.	2023-01-26 11:13:11 +00:00
Jonathan Davies	f6eaf3661c	garage/Cargo.toml: Updated timeage from 0.3 to 0.4.	2023-01-26 11:13:11 +00:00
Jonathan Davies	d3b2a68988	{garage,util}/Cargo.toml: Updated toml from 0.5 to 0.6.	2023-01-26 11:13:11 +00:00
Jonathan Davies	36944f1839	Cargo.toml: Updated base64 from 0.13 to 0.21.	2023-01-26 11:13:07 +00:00
Jonathan Davies	4962b88f8b	tests/s3/website.rs: Added website hosting authorization check tests.	2023-01-13 09:39:02 +00:00
Alex	dac254a6e7	Merge branch 'main' into k2v-watch-range-2	2023-01-11 17:09:37 +01:00
Alex	de1111076b	PollRange integration test	2023-01-11 10:04:41 +01:00
Jonathan Davies	cb07e6145c	Changed all instances of assignation to assignment.	2023-01-05 11:09:25 +00:00
Felix Scheinost	f2106c2733	Implement rpc_secret_file	2023-01-04 18:35:10 +01:00
Alex	29dbcb8278	bg var operation on all nodes at once	2023-01-04 13:25:57 +01:00
Alex	f3f27293df	Uniform framework for bg variable management	2023-01-04 13:07:13 +01:00
Alex	80e4abb98d	Merge pull request 'Changed all instances of 'key new' to 'key create' to make it the same as the bucket commands.' (#459) from jpds/garage:key-create-standardize into next ... Reviewed-on: #459	2023-01-04 10:35:49 +00:00
Alex	570e5e5bbb	Merge branch 'main' into next	2023-01-04 11:34:43 +01:00
Alex	8d5505514f	Make it explicit when using nonversioned encoding	2023-01-03 15:27:36 +01:00
Alex	cdb2a591e9	Refactor how things are migrated	2023-01-03 14:44:47 +01:00
Jonathan Davies	8be862aa19	Changed all instances of 'key new' to 'key create' to make it consistent as bucket commands issued normally around the same time.	2023-01-03 11:11:12 +00:00
Alex	939a6d67e8	Merge branch 'main' into internals-rework	2023-01-02 15:07:44 +01:00
Alex	6775569525	Bump everything to v0.8.1	2023-01-02 14:15:33 +01:00
Alex	6b857a9b8c	cargo fmt	2023-01-02 13:50:42 +01:00
Alex	1fcd0b371b	online repair workers: retry on error	2022-12-14 16:31:31 +01:00
Alex	dfc131850a	Simplified and more aggressive worker exit logic	2022-12-14 15:25:29 +01:00
Alex	d56c472712	Refactor background runner and get rid of job worker	2022-12-14 12:51:42 +01:00
Alex	2183518edc	Spawn all background workers in a separate step	2022-12-14 12:28:07 +01:00
Alex	d1279e04f3	Fix error messages	2022-12-13 16:18:01 +01:00
Alex	f8d5409894	cli: more info displayed on error in garage stats	2022-12-13 15:46:04 +01:00
Alex	d6040e32a6	cli: prettier table in garage stats	2022-12-13 15:43:22 +01:00
Alex	d7f90cabb0	Implement block retry-now and block purge	2022-12-13 15:02:42 +01:00
Alex	687660b27f	Implement block list-errors and block info	2022-12-13 14:23:45 +01:00
Alex	9d82196945	cli: new worker info command	2022-12-13 12:24:30 +01:00
Alex	a51e8d94c6	cli: rename resync-n-workers into resync-worker-count	2022-12-13 11:44:11 +01:00
Alex	de9d6cddf7	Prettier worker list table; remove useless CLI log messages	2022-12-12 17:17:05 +01:00
Alex	0e61e3b6fb	Fix bucket creation tests to take permissions into account	2022-12-12 15:47:55 +01:00
Alex	2c2e65ad8b	Merge commit 'ec12d6c' into next	2022-12-11 18:41:15 +01:00
Alex	26121bb619	Fix logs appearing twice	2022-11-29 22:23:27 +01:00
Alex	ec12d6c8dd	Slightly simplify code at places	2022-11-08 16:15:45 +01:00
Alex	fc2729cd81	Fix integration test	2022-11-08 15:19:46 +01:00
Alex	d75b37b018	Return more info when layout's .check() fails, fix compilation, fix test	2022-11-08 14:58:39 +01:00
Alex	73a4ca8b15	Use bytes as capacity units	2022-11-07 21:12:11 +01:00
Alex	ea5afc2511	Style improvements	2022-11-07 20:11:30 +01:00
Alex	28d7a49f63	Merge branch 'main' into optimal-layout	2022-11-07 12:20:59 +01:00
Alex	e03d9062f7	Show a nice message and a backtrace when Garage panics	2022-11-04 16:39:02 +01:00
Alex	8d3bbf5703	Clearer error messsages	2022-11-04 16:07:33 +01:00
Alex	5b18fd8201	Add garage bucket cleanup-incomplete-uploads command	2022-11-04 11:55:59 +01:00
Alex	002b9fc50c	Add TLS support for Consul discovery + refactoring	2022-10-18 18:38:20 +02:00
Alex	5670599372	Merge pull request 'Use status code 204 No Content for empty responses' (#403) from tobikris/garage:http-no-content into main ... Reviewed-on: #403	2022-10-18 14:20:44 +00:00
Tobias Krischer	7865003323	Use status code 204 No Content for empty responses	2022-10-17 10:55:26 +02:00
Alex	fcaee3bea0	definitively expunge openssl from dependencies everywhere	2022-10-14 18:10:36 +02:00
Mendes	e5664c9822	Improved the statistics displayed in layout show ... corrected a few bugs	2022-10-11 17:17:13 +02:00
Mendes	4abab246f1	cargo fmt	2022-10-10 17:21:13 +02:00
Mendes	fcf9ac674a	Tests written in layout.rs ... added staged_parameters to ClusterLayout removed the serde(default) -> will need a migration function	2022-10-10 17:19:25 +02:00
Mendes	9407df60cc	Corrected two bugs: ... - self.node_id_vec was not properly updated when the previous ring was empty - ClusterLayout::merge was not considering changes in the layout parameters	2022-10-06 12:54:51 +02:00
Mendes	a951b6c452	Added a CLI command to update the parameters for the layout computation (for now, only the zone redundancy)	2022-10-05 16:04:19 +02:00
Mendes	ceac3713d6	modifications in several files to : ... - have consistent error return types - store the zone redundancy in a Lww - print the error and message in the CLI (TODO: for the server Api, should msg be returned in the body response?)	2022-10-05 15:29:48 +02:00
Alex	1f97ce37e6	Shutdown properly on SIGTERM/SIGHUP and on Windows signals	2022-09-28 10:41:59 +02:00
Alex	782630fc27	Initialize metrics exporter earlier (fix #389)	2022-09-20 17:50:22 +02:00
Alex	76f42a1a2b	Properly return HTTP 204 when deleting non-existent object (fix #227)	2022-09-14 17:07:55 +02:00
Alex	e46dc2a8ef	Allow for hostnames in bootstrap_peers and rpc_public_addr (fix #353)	2022-09-14 16:09:38 +02:00
Alex	28a4af73ca	Use netapp 0.5 published from crates.io	2022-09-13 13:11:44 +02:00
Alex	7f54706b95	Merge branch 'lx-perf-improvements' into netapp-stream-body	2022-09-08 15:50:56 +02:00
Alex	d9d199a6c9	Merge branch 'main' into lx-perf-improvements	2022-09-08 15:49:17 +02:00
Alex	ceb1f0229a	Move version back into util	2022-09-07 18:36:46 +02:00
Alex	f310fce34b	Inject GIT_VERSION even later	2022-09-07 18:30:15 +02:00
Alex	06df301de5	Fix merge	2022-09-07 18:16:01 +02:00
Alex	8adc654713	Merge branch 'main' into improve-deps	2022-09-07 18:13:27 +02:00
Alex	107853334b	Fix build error	2022-09-07 18:10:19 +02:00
Alex	1449204439	Add warnings when features are not included in build	2022-09-07 18:02:13 +02:00
Alex	2e00809af5	Error messages when system-libs XOR bundled-libs != 1	2022-09-07 17:57:12 +02:00
Alex	2559f63e9b	Make all HTTP services optionnal	2022-09-07 17:54:16 +02:00
Alex	28d86e7602	Report build features in garage --help	2022-09-07 17:05:21 +02:00
Alex	db61f41030	Move GIT_VERSION injection later in build chain to reduce build times	2022-09-07 11:59:56 +02:00
Alex	6b958979bd	Merge branch 'lx-perf-improvements' into netapp-stream-body	2022-09-06 22:13:01 +02:00
Alex	1e92e9f782	Disable k2v tests when feature is disabled	2022-09-06 17:29:46 +02:00
Alex	431dee050f	Remove opentelemetry-otlp dep in api/	2022-09-06 17:25:44 +02:00
Alex	b886c75450	Make all DB engines optional build features	2022-09-06 17:09:43 +02:00
Alex	48ffaaadfc	Bump versions to 0.8.0 (compatibility is broken already)	2022-09-06 16:47:56 +02:00
Alex	454d8474ef	Fix clippy	2022-09-06 15:43:50 +02:00
Jakub Jirutka	ea36b9ff90	Allow building without Prometheus exporter (/metrics endpoint) ... prometheus and opentelemetry-prometheus add 7 extra dependencies in total and increases the size of the garage binary by ~7 % (with fat LTO).	2022-09-06 01:15:09 +02:00
Jakub Jirutka	e7af006c1c	Make OTLP exporter optional via feature "telemetry-otlp" ... opentelemetry-otlp add 48 (!) extra dependencies and increases the size of the garage binary by ~11 % (with fat LTO).	2022-09-06 01:14:47 +02:00
Alex	07e6bcde85	Merge branch 'main' into lx-perf-improvements	2022-09-05 12:40:17 +02:00
Jakub Jirutka	7511ba5530	Allow linking against system-provided libsqlite ... Unfortunately, rusqlite uses the opposite logic for enabling/disabling bundled libraries to others (libsodium-sys, zstd-sys). Cargo features are very limited and doesn't allow to enable feature A in a dependency iff feature B is disabled. Note, lmdb-rkv-sys doesn't need any special treatment because it automatically links against system liblmdb if found via pkgconf. Linux distros should build garage with `--no-default-features --features system-libs` to disable bundled-libs and enable system-libs.	2022-09-03 19:15:57 +02:00
Jakub Jirutka	a6e40b75ea	Add feature "system-libs" to enable linking against system libraries ... If this feature is enabled, libsodium-sys and zstd-sys will link dynamically against system-provided libraries instead of building and linking statically the bundled (possibly outdated and vulnerable) copies of them. This feature is intended mainly for linux package maintainers.	2022-09-03 18:44:34 +02:00
Alex	5d4b937a00	Ability to have up to 4 concurrently working resync workers	2022-09-02 17:18:13 +02:00
Alex	47be652a1f	block manager: refactor: split resync into separate file	2022-09-02 16:47:15 +02:00
Alex	943d76c583	Ability to dynamically set resync tranquility	2022-09-02 15:34:21 +02:00
Alex	c9bc9d89de	Merge branch 'lx-perf-improvements' into netapp-stream-body	2022-08-31 17:42:31 +02:00
Alex	efbca67ce4	Add env filter to tracing subscriber	2022-08-31 14:39:12 +02:00
Alex	44cd98d2e4	Tracing-subscriber: write to stderr	2022-08-31 14:28:17 +02:00
Alex	dd5304f6fc	Replace logging crate pretty_env_logger by tracing_subscriber::fmt	2022-08-31 14:24:41 +02:00
Alex	1921f4f7e6	Merge branch 'lx-perf-improvements' into netapp-stream-body	2022-08-29 16:45:05 +02:00