withinboredom/garage
1
0
Fork 0
forked from Deuxfleurs/garage
Code Issues Pull requests Projects Releases Wiki Activity
731 commits 11 branches 0 tags 116 MiB
Commit graph

731 commits

This branch
This branch
All branches
Author SHA1 Message Date
Mendes
7f3249a237 New version of the algorithm that calculate the layout. 
It takes as paramters the replication factor and the zone redundancy, computes the
largest partition size reachable with these constraints, and among the possible
assignation with this partition size, it computes the one that moves the least number
of partitions compared to the previous assignation.
This computation uses graph algorithms defined in graph_algo.rs
 2022-09-21 14:39:59 +02:00
Mendes
c4adbeed51 Added the section with description proofs of the parametric assignment computation in the optimal layout report 2022-09-10 13:51:12 +02:00
Mendes
d38fb6c250 ignore log files in commit 2022-09-08 12:43:33 +02:00
Mendes
81083dd415 Added a first draft version of the algorithm and analysis for the non-strict mode. 2022-08-19 21:21:41 +02:00
Mendes
7b2c065c82 Merge branch 'optimal-layout' of https://git.deuxfleurs.fr/Deuxfleurs/garage into optimal-layout 2022-07-19 13:30:49 +02:00
Mendes
03e3a1bd15 Added the latex report on the optimal layout algorithm 2022-07-18 22:35:29 +02:00
Alex Auvolat
617f28bfa4
Correct small formatting issue 2022-05-05 14:21:57 +02:00
Mendes
948ff93cf1 Corrected the warnings and errors issued by cargo clippy 2022-05-01 16:05:39 +02:00
Alex Auvolat
3ba2c5b424
updated cargo.lock 2022-05-01 10:11:43 +02:00
Alex Auvolat
2aeaddd5e2
Apply cargo fmt 2022-05-01 09:57:05 +02:00
Alex Auvolat
c1d1646c4d
Change the way new layout assignations are computed. 
The function now computes an optimal assignation (with respect to partition size) that minimizes the distance to the former assignation, using flow algorithms.

This commit was written by Mendes Oulamara <mendes.oulamara@pm.me>
 2022-05-01 09:54:19 +02:00
Alex Auvolat
c9ef3e461b
fix clippy 2022-04-19 12:50:40 +02:00
Alex Auvolat
c93008d333
Prettier code for defragmentation 2022-04-19 12:50:40 +02:00
Alex Auvolat
e5341ca47b
Defragmentation in UploadPartCopy: first pass (not pretty but it compiles) 2022-04-19 12:50:36 +02:00
withinboredom
a4f9f19ac3 remove size limitation in UploadPartCopy (#280) 
This removes the >1mb s3_copy restriction.

This restriction doesn't seem to be documented anywhere (I could be wrong). It also causes some software to fail (such as #248).

Co-authored-by: Rob Landers <landers.robert@gmail.com>
Reviewed-on: Deuxfleurs/garage#280
Co-authored-by: withinboredom <landers.robert@gmail.com>
Co-committed-by: withinboredom <landers.robert@gmail.com>
 2022-04-19 12:49:43 +02:00
Baptiste Jonglez
47e57518ec Add documentation on running Kopia with Garage 2022-04-10 13:04:07 +02:00
Alex Auvolat
dffcd9f4b1
update Cargo.nix 2022-04-08 14:35:09 +02:00
Alex Auvolat
5d404dcd54
Add missing opentelemetry features 2022-04-08 14:21:04 +02:00
Quentin Dufour
62f0715abe Add/Fix OpenTelemetry 2022-04-07 16:12:35 +02:00
Quentin Dufour
7e1ac51b58 Add files to quickly test k8s 2022-04-07 16:12:35 +02:00
Alex Auvolat
94f1e48fff Update to netapp 0.4.2 (a tiny fix) 2022-04-07 11:50:03 +02:00
Alex Auvolat
cb5836d53c Bring maximum exponential backoff time down from 16h to 1h 2022-04-07 11:49:29 +02:00
Quentin Dufour
8e3ee82c3e Be clearer on what upgrades are (not) supported 2022-04-06 21:45:59 +02:00
Quentin Dufour
a122a8cb46 Add an "upgrading" section, add a guide for 0.7 2022-04-05 10:08:31 +02:00
Quentin Dufour
9fd8ec1dee Add documentation for winscp+sftpgo 2022-03-31 10:25:56 +02:00
Alex Auvolat
0091002ef2
New replication modes and their documentation 2022-03-28 16:26:04 +02:00
Alex Auvolat
8f9cf3a5d1
fix a clippy lint 2022-03-28 15:48:55 +02:00
Alex Auvolat
913f7754bb
Add blocks in errored state to garage stats 2022-03-28 15:47:23 +02:00
Alex Auvolat
42dde54126
Log admin GET requests at debug level instead of info 
to reduce noise in logs
 2022-03-28 15:46:52 +02:00
Alex Auvolat
dca2ffdf91
document administrative options 2022-03-28 12:26:08 +02:00
Quentin Dufour
0cf4efac89 Compile kuberetes-discovery only when release=true 2022-03-24 16:57:43 +01:00
Alex Auvolat
9d0ed78887 Add feature flag for Kubernetes discovery 2022-03-24 16:57:43 +01:00
Alex Auvolat
509d256c58
Make layout optimization work in relative terms 2022-03-24 15:27:14 +01:00
Alex Auvolat
2814d41842
Allow garage layout assign to assign to several nodes at once 2022-03-24 15:27:13 +01:00
Alex Auvolat
7e0e2ffda2
Slight change and add comment to layout assignation algo 2022-03-24 15:27:13 +01:00
Alex Auvolat
413ab0eaed
Small change to partition assignation algorithm 
This change helps ensure that nodes for each partition are spread
over all datacenters, a property that wasn't ensured previously
when going from a 2 DC deployment to a 3 DC deployment
 2022-03-24 15:27:10 +01:00
Alex Auvolat
43945234ae
Add missing src/block to toplevel cargo.toml 2022-03-23 10:26:10 +01:00
Alex Auvolat
3dc9214172
Add lots of comments on how the resync queue works 
(I don't really want to change/refactor that code though)
 2022-03-23 10:25:39 +01:00
Alex Auvolat
077dd1cde9
Clippy 2022-03-23 10:25:39 +01:00
Alex Auvolat
2d13f0aa13
run cargo2nix 2022-03-23 10:25:37 +01:00
Alex Auvolat
e480aaf338
Make background tranquility a configurable parameter 2022-03-23 10:25:19 +01:00
Alex Auvolat
8fd6745745
Move block RC code to separate rc.rs 2022-03-23 10:25:19 +01:00
Alex Auvolat
c3982a90b6
Move DataBlock out of manager.rs 2022-03-23 10:25:19 +01:00
Alex Auvolat
c1d9854d2c
Move block manager to separate module 2022-03-23 10:25:15 +01:00
trinity-1686a
8565f7dc31 cleanup 2022-03-23 10:22:37 +01:00
trinity-1686a
8db6b84559 add test for create bucket and put website with streaming signature 2022-03-23 10:22:37 +01:00
trinity-1686a
1eb7fdb08f add test framework for arbitraty S3 requests 
and implement some basic test with it
 2022-03-23 10:22:36 +01:00
KokaKiwi
e934934f14 garage_api: Update streaming payload stream unit tests 2022-03-23 10:22:36 +01:00
KokaKiwi
98545a16dd garage_api: Handle streaming payload early in request handling 2022-03-23 10:22:36 +01:00
Alex Auvolat
822128e3c8 Talk a bit about capacity balancing between regions 2022-03-22 12:07:13 +01:00