731 Commits

All Branches

Search

Author	SHA1	Message	Date
Mendes	7f3249a237	New version of the algorithm that calculate the layout. ... It takes as paramters the replication factor and the zone redundancy, computes the largest partition size reachable with these constraints, and among the possible assignation with this partition size, it computes the one that moves the least number of partitions compared to the previous assignation. This computation uses graph algorithms defined in graph_algo.rs	2022-09-21 14:39:59 +02:00
Mendes	c4adbeed51	Added the section with description proofs of the parametric assignment computation in the optimal layout report	2022-09-10 13:51:12 +02:00
Mendes	d38fb6c250	ignore log files in commit	2022-09-08 12:43:33 +02:00
Mendes	81083dd415	Added a first draft version of the algorithm and analysis for the non-strict mode.	2022-08-19 21:21:41 +02:00
Mendes	7b2c065c82	Merge branch 'optimal-layout' of https://git.deuxfleurs.fr/Deuxfleurs/garage into optimal-layout	2022-07-19 13:30:49 +02:00
Mendes	03e3a1bd15	Added the latex report on the optimal layout algorithm	2022-07-18 22:35:29 +02:00
Alex	617f28bfa4	Correct small formatting issue	2022-05-05 14:21:57 +02:00
Mendes	948ff93cf1	Corrected the warnings and errors issued by cargo clippy	2022-05-01 16:05:39 +02:00
Alex	3ba2c5b424	updated cargo.lock	2022-05-01 10:11:43 +02:00
Alex	2aeaddd5e2	Apply cargo fmt	2022-05-01 09:57:05 +02:00
Alex	c1d1646c4d	Change the way new layout assignations are computed. ... The function now computes an optimal assignation (with respect to partition size) that minimizes the distance to the former assignation, using flow algorithms. This commit was written by Mendes Oulamara <mendes.oulamara@pm.me>	2022-05-01 09:54:19 +02:00
Alex	c9ef3e461b	fix clippy	2022-04-19 12:50:40 +02:00
Alex	c93008d333	Prettier code for defragmentation	2022-04-19 12:50:40 +02:00
Alex	e5341ca47b	Defragmentation in UploadPartCopy: first pass (not pretty but it compiles)	2022-04-19 12:50:36 +02:00
withinboredom	a4f9f19ac3	remove size limitation in UploadPartCopy (#280) ... This removes the >1mb s3_copy restriction. This restriction doesn't seem to be documented anywhere (I could be wrong). It also causes some software to fail (such as #248). Co-authored-by: Rob Landers <landers.robert@gmail.com> Reviewed-on: #280 Co-authored-by: withinboredom <landers.robert@gmail.com> Co-committed-by: withinboredom <landers.robert@gmail.com>	2022-04-19 12:49:43 +02:00
Baptiste Jonglez	47e57518ec	Add documentation on running Kopia with Garage	2022-04-10 13:04:07 +02:00
Alex	dffcd9f4b1	update Cargo.nix	2022-04-08 14:35:09 +02:00
Alex	5d404dcd54	Add missing opentelemetry features	2022-04-08 14:21:04 +02:00
Quentin	62f0715abe	Add/Fix OpenTelemetry	2022-04-07 16:12:35 +02:00
Quentin	7e1ac51b58	Add files to quickly test k8s	2022-04-07 16:12:35 +02:00
Alex	94f1e48fff	Update to netapp 0.4.2 (a tiny fix)	2022-04-07 11:50:03 +02:00
Alex	cb5836d53c	Bring maximum exponential backoff time down from 16h to 1h	2022-04-07 11:49:29 +02:00
Quentin	8e3ee82c3e	Be clearer on what upgrades are (not) supported	2022-04-06 21:45:59 +02:00
Quentin	a122a8cb46	Add an "upgrading" section, add a guide for 0.7	2022-04-05 10:08:31 +02:00
Quentin	9fd8ec1dee	Add documentation for winscp+sftpgo	2022-03-31 10:25:56 +02:00
Alex	0091002ef2	New replication modes and their documentation	2022-03-28 16:26:04 +02:00
Alex	8f9cf3a5d1	fix a clippy lint	2022-03-28 15:48:55 +02:00
Alex	913f7754bb	Add blocks in errored state to `garage stats`	2022-03-28 15:47:23 +02:00
Alex	42dde54126	Log admin GET requests at debug level instead of info ... to reduce noise in logs	2022-03-28 15:46:52 +02:00
Alex	dca2ffdf91	document administrative options	2022-03-28 12:26:08 +02:00
Quentin	0cf4efac89	Compile kuberetes-discovery only when release=true	2022-03-24 16:57:43 +01:00
Alex	9d0ed78887	Add feature flag for Kubernetes discovery	2022-03-24 16:57:43 +01:00
Alex	509d256c58	Make layout optimization work in relative terms	2022-03-24 15:27:14 +01:00
Alex	2814d41842	Allow `garage layout assign` to assign to several nodes at once	2022-03-24 15:27:13 +01:00
Alex	7e0e2ffda2	Slight change and add comment to layout assignation algo	2022-03-24 15:27:13 +01:00
Alex	413ab0eaed	Small change to partition assignation algorithm ... This change helps ensure that nodes for each partition are spread over all datacenters, a property that wasn't ensured previously when going from a 2 DC deployment to a 3 DC deployment	2022-03-24 15:27:10 +01:00
Alex	43945234ae	Add missing src/block to toplevel cargo.toml	2022-03-23 10:26:10 +01:00
Alex	3dc9214172	Add lots of comments on how the resync queue works ... (I don't really want to change/refactor that code though)	2022-03-23 10:25:39 +01:00
Alex	077dd1cde9	Clippy	2022-03-23 10:25:39 +01:00
Alex	2d13f0aa13	run cargo2nix	2022-03-23 10:25:37 +01:00
Alex	e480aaf338	Make background tranquility a configurable parameter	2022-03-23 10:25:19 +01:00
Alex	8fd6745745	Move block RC code to separate `rc.rs`	2022-03-23 10:25:19 +01:00
Alex	c3982a90b6	Move DataBlock out of manager.rs	2022-03-23 10:25:19 +01:00
Alex	c1d9854d2c	Move block manager to separate module	2022-03-23 10:25:15 +01:00
trinity-1686a	8565f7dc31	cleanup	2022-03-23 10:22:37 +01:00
trinity-1686a	8db6b84559	add test for create bucket and put website with streaming signature	2022-03-23 10:22:37 +01:00
trinity-1686a	1eb7fdb08f	add test framework for arbitraty S3 requests ... and implement some basic test with it	2022-03-23 10:22:36 +01:00
Jill	e934934f14	garage_api: Update streaming payload stream unit tests	2022-03-23 10:22:36 +01:00
Jill	98545a16dd	garage_api: Handle streaming payload early in request handling	2022-03-23 10:22:36 +01:00
Alex	822128e3c8	Talk a bit about capacity balancing between regions	2022-03-22 12:07:13 +01:00