485 commits

Author	SHA1	Message	Date
Alex Auvolat	94f1e48fff	Update to netapp 0.4.2 (a tiny fix)	2022-04-07 11:50:03 +02:00
Alex Auvolat	cb5836d53c	Bring maximum exponential backoff time down from 16h to 1h	2022-04-07 11:49:29 +02:00
Alex Auvolat	0091002ef2	New replication modes and their documentation	2022-03-28 16:26:04 +02:00
Alex Auvolat	8f9cf3a5d1	fix a clippy lint	2022-03-28 15:48:55 +02:00
Alex Auvolat	913f7754bb	Add blocks in errored state to garage stats	2022-03-28 15:47:23 +02:00
Alex Auvolat	42dde54126	Log admin GET requests at debug level instead of info ... to reduce noise in logs	2022-03-28 15:46:52 +02:00
Alex Auvolat	9d0ed78887	Add feature flag for Kubernetes discovery	2022-03-24 16:57:43 +01:00
Alex Auvolat	509d256c58	Make layout optimization work in relative terms	2022-03-24 15:27:14 +01:00
Alex Auvolat	2814d41842	Allow garage layout assign to assign to several nodes at once	2022-03-24 15:27:13 +01:00
Alex Auvolat	7e0e2ffda2	Slight change and add comment to layout assignation algo	2022-03-24 15:27:13 +01:00
Alex Auvolat	413ab0eaed	Small change to partition assignation algorithm ... This change helps ensure that nodes for each partition are spread over all datacenters, a property that wasn't ensured previously when going from a 2 DC deployment to a 3 DC deployment	2022-03-24 15:27:10 +01:00
Alex Auvolat	3dc9214172	Add lots of comments on how the resync queue works ... (I don't really want to change/refactor that code though)	2022-03-23 10:25:39 +01:00
Alex Auvolat	077dd1cde9	Clippy	2022-03-23 10:25:39 +01:00
Alex Auvolat	e480aaf338	Make background tranquility a configurable parameter	2022-03-23 10:25:19 +01:00
Alex Auvolat	8fd6745745	Move block RC code to separate rc.rs	2022-03-23 10:25:19 +01:00
Alex Auvolat	c3982a90b6	Move DataBlock out of manager.rs	2022-03-23 10:25:19 +01:00
Alex Auvolat	c1d9854d2c	Move block manager to separate module	2022-03-23 10:25:15 +01:00
trinity-1686a	8565f7dc31	cleanup	2022-03-23 10:22:37 +01:00
trinity-1686a	8db6b84559	add test for create bucket and put website with streaming signature	2022-03-23 10:22:37 +01:00
trinity-1686a	1eb7fdb08f	add test framework for arbitraty S3 requests ... and implement some basic test with it	2022-03-23 10:22:36 +01:00
KokaKiwi	e934934f14	garage_api: Update streaming payload stream unit tests	2022-03-23 10:22:36 +01:00
KokaKiwi	98545a16dd	garage_api: Handle streaming payload early in request handling	2022-03-23 10:22:36 +01:00
Alex Auvolat	db46cdef79	Update netapp to v0.4.1	2022-03-15 17:09:57 +01:00
Alex Auvolat	ba6b56ae68	Fix some new clippy lints	2022-03-14 12:27:49 +01:00
Alex Auvolat	0af314b295	Add comment for fsync	2022-03-14 11:54:00 +01:00
Alex Auvolat	d78bf379fb	Fix resync queue to not drop items	2022-03-14 11:51:37 +01:00
Alex Auvolat	f7e6f4616f	Spawn a single resync worker	2022-03-14 11:51:37 +01:00
Alex Auvolat	dc5ec4ecf9	Add appropriate fsync() calls in write_block ... to ensure that data is persisted properly	2022-03-14 11:51:32 +01:00
Alex Auvolat	fe62d01b7e	Implement exponential backoff for resync retries	2022-03-14 11:41:20 +01:00
Alex Auvolat	9b2b531f4d	Make admin server optional	2022-03-14 10:54:25 +01:00
Alex Auvolat	2377a92f6b	Add wrapper over sled tree to count items (used for big queues)	2022-03-14 10:54:25 +01:00
Alex Auvolat	203e8d2c34	Bump version to 0.7 because of incompatible Netapp	2022-03-14 10:54:24 +01:00
Alex Auvolat	f869ca625d	Add spans to table calls, change span names in RPC	2022-03-14 10:54:12 +01:00
Alex Auvolat	0cc31ee169	add missing netapp telemetry feature	2022-03-14 10:54:11 +01:00
Alex Auvolat	dc8d0496cc	Refactoring: rename config files, make modifications less invasive	2022-03-14 10:53:51 +01:00
Alex Auvolat	d9a35359bf	Add metrics to web endpoint	2022-03-14 10:53:50 +01:00
Alex Auvolat	2a5609b292	Add metrics to API endpoint	2022-03-14 10:53:36 +01:00
Alex Auvolat	818daa5c78	Refactor how durations are measured	2022-03-14 10:53:35 +01:00
Alex Auvolat	f0d0cd9a20	Remove strum crate dependency; add protobuf nix dependency	2022-03-14 10:53:00 +01:00
Alex Auvolat	55d4471599	Remove ... at end of hex IDs	2022-03-14 10:52:31 +01:00
Alex Auvolat	bb04d94fa9	Update to Netapp 0.4 which supports distributed tracing	2022-03-14 10:52:30 +01:00
Alex Auvolat	8c2fb0c066	Add tracing integration with opentelemetry	2022-03-14 10:52:13 +01:00
Alex Auvolat	2cab84b1fe	Add many metrics in table/ and rpc/	2022-03-14 10:51:50 +01:00
Maximilien R	1e2cf26373	Implement basic metrics in table	2022-03-14 10:51:17 +01:00
mricher	e349af13a7	Update dependencies and add admin module with metrics ... - Global dependencies updated in Cargo.lock - New module created in src/admin to host: - the (future) admin REST API - the metric collection - add configuration block No metrics implemented yet	2022-03-14 10:51:12 +01:00
Max Audron	9d44127245	add support for kubernetes service discovery ... This commit adds support to discover garage instances running in kubernetes. Once enabled by setting `kubernetes_namespace` and `kubernetes_service_name` garage will create a Custom Resources `garagenodes.deuxfleurs.fr` with nodes public key as the resource name. and IP and Port information as spec in the namespace configured by `kubernetes_namespace`. For discovering nodes the resources are filtered with the optionally set `kubernetes_service_name` which sets a label `garage.deuxfleurs.fr/service` on the resources. This allows to separate multiple garage deployments in a single namespace. the `kubernetes_skip_crd` variable allows to disable the creation of the CRD by garage itself. The user must deploy this manually.	2022-03-12 13:05:52 +01:00
Quentin Dufour	c00b2c9948	Functional tests for admin commands	2022-03-07 17:32:07 +01:00
Quentin Dufour	8df1e186de	Functional tests for website endpoints	2022-03-07 17:32:07 +01:00
Quentin Dufour	2ef60b8417	Functional test for multipart endpoints	2022-03-07 17:32:07 +01:00
Quentin Dufour	1e639ec67c	Functional test for ListMultipartUploads	2022-03-07 17:32:07 +01:00