Commit graph

1971 commits

Author SHA1 Message Date
c1d1646c4d
Change the way new layout assignations are computed.
Some checks failed
continuous-integration/drone/push Build is failing
The function now computes an optimal assignation (with respect to partition size) that minimizes the distance to the former assignation, using flow algorithms.

This commit was written by Mendes Oulamara <mendes.oulamara@pm.me>
2022-05-01 09:54:19 +02:00
c9ef3e461b
fix clippy
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-04-19 12:50:40 +02:00
c93008d333
Prettier code for defragmentation 2022-04-19 12:50:40 +02:00
e5341ca47b
Defragmentation in UploadPartCopy: first pass (not pretty but it compiles) 2022-04-19 12:50:36 +02:00
a4f9f19ac3 remove size limitation in UploadPartCopy (#280)
All checks were successful
continuous-integration/drone/push Build is passing
This removes the >1mb s3_copy restriction.

This restriction doesn't seem to be documented anywhere (I could be wrong). It also causes some software to fail (such as #248).

Co-authored-by: Rob Landers <landers.robert@gmail.com>
Reviewed-on: #280
Co-authored-by: withinboredom <landers.robert@gmail.com>
Co-committed-by: withinboredom <landers.robert@gmail.com>
2022-04-19 12:49:43 +02:00
Baptiste Jonglez
47e57518ec Add documentation on running Kopia with Garage
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-04-10 13:04:07 +02:00
dffcd9f4b1
update Cargo.nix
All checks were successful
continuous-integration/drone/tag Build is passing
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
continuous-integration/drone Build is passing
2022-04-08 14:35:09 +02:00
5d404dcd54
Add missing opentelemetry features 2022-04-08 14:21:04 +02:00
62f0715abe Add/Fix OpenTelemetry
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-04-07 16:12:35 +02:00
7e1ac51b58 Add files to quickly test k8s 2022-04-07 16:12:35 +02:00
94f1e48fff Update to netapp 0.4.2 (a tiny fix)
All checks were successful
continuous-integration/drone/push Build is passing
2022-04-07 11:50:03 +02:00
cb5836d53c Bring maximum exponential backoff time down from 16h to 1h
All checks were successful
continuous-integration/drone/push Build is passing
2022-04-07 11:49:29 +02:00
8e3ee82c3e Be clearer on what upgrades are (not) supported
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-04-06 21:45:59 +02:00
a122a8cb46 Add an "upgrading" section, add a guide for 0.7
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2022-04-05 10:08:31 +02:00
9fd8ec1dee Add documentation for winscp+sftpgo
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-03-31 10:25:56 +02:00
0091002ef2
New replication modes and their documentation
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-03-28 16:26:04 +02:00
8f9cf3a5d1
fix a clippy lint
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-03-28 15:48:55 +02:00
913f7754bb
Add blocks in errored state to garage stats
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2022-03-28 15:47:23 +02:00
42dde54126
Log admin GET requests at debug level instead of info
to reduce noise in logs
2022-03-28 15:46:52 +02:00
dca2ffdf91
document administrative options
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-03-28 12:26:08 +02:00
0cf4efac89 Compile kuberetes-discovery only when release=true
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
continuous-integration/drone Build is passing
2022-03-24 16:57:43 +01:00
9d0ed78887 Add feature flag for Kubernetes discovery 2022-03-24 16:57:43 +01:00
509d256c58
Make layout optimization work in relative terms
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-03-24 15:27:14 +01:00
2814d41842
Allow garage layout assign to assign to several nodes at once 2022-03-24 15:27:13 +01:00
7e0e2ffda2
Slight change and add comment to layout assignation algo 2022-03-24 15:27:13 +01:00
413ab0eaed
Small change to partition assignation algorithm
This change helps ensure that nodes for each partition are spread
over all datacenters, a property that wasn't ensured previously
when going from a 2 DC deployment to a 3 DC deployment
2022-03-24 15:27:10 +01:00
43945234ae
Add missing src/block to toplevel cargo.toml
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-03-23 10:26:10 +01:00
3dc9214172
Add lots of comments on how the resync queue works
(I don't really want to change/refactor that code though)
2022-03-23 10:25:39 +01:00
077dd1cde9
Clippy 2022-03-23 10:25:39 +01:00
2d13f0aa13
run cargo2nix 2022-03-23 10:25:37 +01:00
e480aaf338
Make background tranquility a configurable parameter 2022-03-23 10:25:19 +01:00
8fd6745745
Move block RC code to separate rc.rs 2022-03-23 10:25:19 +01:00
c3982a90b6
Move DataBlock out of manager.rs 2022-03-23 10:25:19 +01:00
c1d9854d2c
Move block manager to separate module 2022-03-23 10:25:15 +01:00
8565f7dc31 cleanup
All checks were successful
continuous-integration/drone/push Build is passing
2022-03-23 10:22:37 +01:00
8db6b84559 add test for create bucket and put website with streaming signature 2022-03-23 10:22:37 +01:00
1eb7fdb08f add test framework for arbitraty S3 requests
and implement some basic test with it
2022-03-23 10:22:36 +01:00
e934934f14 garage_api: Update streaming payload stream unit tests 2022-03-23 10:22:36 +01:00
98545a16dd garage_api: Handle streaming payload early in request handling 2022-03-23 10:22:36 +01:00
822128e3c8 Talk a bit about capacity balancing between regions
All checks were successful
continuous-integration/drone/push Build is passing
2022-03-22 12:07:13 +01:00
Rune Henriksen
aea8b41728 document request routing logic
All checks were successful
continuous-integration/drone/push Build is passing
2022-03-21 12:03:57 +01:00
Rune Henriksen
71e6645e09 add short tutorial for duplicati usage with garage
All checks were successful
continuous-integration/drone/push Build is passing
2022-03-21 11:58:19 +01:00
15da2156f6 Change position of the node-id argument
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-03-19 18:03:23 +01:00
0529f3c34d Patch cargo2nix openssl override
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone Build is passing
continuous-integration/drone/push Build is passing
2022-03-17 12:17:38 +01:00
db46cdef79
Update netapp to v0.4.1
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-03-15 17:09:57 +01:00
ba6b56ae68
Fix some new clippy lints
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-03-14 12:27:49 +01:00
0af314b295
Add comment for fsync
Some checks reported errors
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build was killed
2022-03-14 11:54:00 +01:00
d78bf379fb
Fix resync queue to not drop items
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2022-03-14 11:51:37 +01:00
f7e6f4616f
Spawn a single resync worker 2022-03-14 11:51:37 +01:00
dc5ec4ecf9
Add appropriate fsync() calls in write_block
to ensure that data is persisted properly
2022-03-14 11:51:32 +01:00