KokaKiwi/garage
1
0
Fork 0
forked from Deuxfleurs/garage
Code Issues Pull requests Projects Releases Wiki Activity
1369 commits 13 branches 1 tag 58 MiB
Commit graph

842 commits

Author SHA1 Message Date
Alex Auvolat
656b8d42de secrets can be passed directly in config, as file, or as env 2023-02-03 15:27:39 +01:00
Jonathan Davies
1b6ec74748 error.rs: Corrected error messages to say unexpected scope. 2023-02-02 16:20:31 +00:00
Alex
f9573b6912 Merge pull request 'Fix duplicated content-type in error document' (#493) from baptiste/garage:fix_error_document_content_type into main 
Reviewed-on: Deuxfleurs/garage#493
 2023-01-30 10:56:35 +00:00
Baptiste Jonglez
e85a200189 Fix duplicated content-type in error document 
Fixes #492
 2023-01-29 22:51:23 +01:00
Jonathan Davies
9c354f0a8f Improved bucket authorization response strings. 2023-01-29 20:34:41 +00:00
Jonathan Davies
004bb5b4f1 api_server.rs: Adapted to use query string per Caddy upstream change. 2023-01-29 20:34:37 +00:00
Alex Auvolat
d6af95d205 fix cli display bug 2023-01-26 17:50:50 +01:00
Alex Auvolat
c56794655e Fix fmt 2023-01-26 17:27:03 +01:00
Alex Auvolat
8e93d69974 More clippy fixes 2023-01-26 17:26:32 +01:00
Alex Auvolat
3113f6b5f2 more fixes 2023-01-26 17:14:17 +01:00
Alex Auvolat
1dff62564f fix clippy 2023-01-26 17:05:31 +01:00
Alex Auvolat
590a0a8450 Merge branch 'main' into k2v-watch-range-2 2023-01-26 16:46:40 +01:00
Alex Auvolat
94d559ae00 Merge branch 'main' into report-disk-usage 2023-01-26 16:20:41 +01:00
Alex Auvolat
c7d0ad0aa0 Add local disk usage to exported prometheus metrics 2023-01-26 15:30:36 +01:00
Alex Auvolat
efb6b6e868 Disk space report 
Report available disk space on nodes and calculate cluster-wide available space in `garage stats` (fix #479)
 2023-01-26 15:04:32 +01:00
Jonathan Davies
3dc655095f db/Cargo.toml: Updated rusqlite from 0.27 to 0.28. 2023-01-26 11:13:11 +00:00
Jonathan Davies
20c1cdf662 Cargo.toml: Loosen tracing dependency to just 0.1. 2023-01-26 11:13:11 +00:00
Jonathan Davies
f952e37ba7 {model,util}/Cargo.toml: Updated blake2 from 0.9 to 0.10. 2023-01-26 11:13:11 +00:00
Jonathan Davies
fbafa76284 {db,util}/Cargo.toml: Updated mktemp from 0.4 to 0.5. 2023-01-26 11:13:11 +00:00
Jonathan Davies
63e22e71f2 api/Cargo.toml: Updated idna from 0.2 to 0.3. 2023-01-26 11:13:11 +00:00
Jonathan Davies
f6eaf3661c garage/Cargo.toml: Updated timeage from 0.3 to 0.4. 2023-01-26 11:13:11 +00:00
Jonathan Davies
d3b2a68988 {garage,util}/Cargo.toml: Updated toml from 0.5 to 0.6. 2023-01-26 11:13:11 +00:00
Jonathan Davies
b4a1a6a32f util/time.rs: Updated deprecated associated function to timestamp_opt(). 2023-01-26 11:13:11 +00:00
Jonathan Davies
bcac889f9a Cargo.toml: Updated clap from 3.1.18 to 4.1. 2023-01-26 11:13:11 +00:00
Jonathan Davies
9e08a05e69 k2v-client/Cargo.toml: Loosen dependencies. 2023-01-26 11:13:11 +00:00
Jonathan Davies
36944f1839 Cargo.toml: Updated base64 from 0.13 to 0.21. 2023-01-26 11:13:07 +00:00
Jonathan Davies
db56d4658f util/Cargo.toml: Updated rmp-serde from 0.15 to 1.1. 2023-01-26 11:03:43 +00:00
Jonathan Davies
93c3f8fc8c api/Cargo.toml: Updated url from 2.1 to 2.3. 2023-01-23 19:16:58 +00:00
Jonathan Davies
1c435fce09 api/Cargo.toml: Updated httpdate from 0.3 to 1.0. 2023-01-23 19:16:54 +00:00
Jonathan Davies
dead123892 api/Cargo.toml: Updated pin-project to 1.0.12. 2023-01-23 18:39:35 +00:00
Jonathan Davies
5c3075fe01 Cargo.toml: Updated zstd from 0.9 to 0.12. 2023-01-23 18:08:14 +00:00
Jonathan Davies
4962b88f8b tests/s3/website.rs: Added website hosting authorization check tests. 2023-01-13 09:39:02 +00:00
Jonathan Davies
100b01e859 Implemented website hosting authorization endpoint. 
Fixes: #468
 2023-01-13 09:38:58 +00:00
Alex Auvolat
84b4a868e3
Migration of cluster layout from v0.8 to v0.9 2023-01-11 17:47:46 +01:00
Alex Auvolat
dac254a6e7
Merge branch 'main' into k2v-watch-range-2 2023-01-11 17:09:37 +01:00
Alex
94d723f27c Merge pull request 'Implement rpc_secret_file' (#466) from felix.scheinost/garage:feature/implement-rpc-secret-file into main 
Reviewed-on: Deuxfleurs/garage#466
 2023-01-11 16:04:35 +00:00
Alex Auvolat
638c5a3ce0
PollRange: add extra RPC delay after quorum is achieved, 
to give a chance to the 3rd node to respond
 2023-01-11 16:12:07 +01:00
Alex Auvolat
5b5ca63cf6
Poll cleanup 2023-01-11 15:17:27 +01:00
Alex Auvolat
cbfae673e8
PollRange & PollItem: min timeout = 1 sec 2023-01-11 15:03:08 +01:00
Alex Auvolat
bba13f40fc
Correctly return bad requests when seeh marker is invalid 2023-01-11 12:27:19 +01:00
Alex Auvolat
ba384e61c0
PollRange: return immediately if no seen marker is provided 2023-01-11 12:03:17 +01:00
Alex Auvolat
09a3dad0f2
Lock once for insert_many 2023-01-11 11:35:36 +01:00
Alex Auvolat
32aab06929
k2v-client libary poll_range and CLI poll-range 2023-01-11 11:14:29 +01:00
Alex Auvolat
de1111076b
PollRange integration test 2023-01-11 10:04:41 +01:00
Alex Auvolat
b83517d521
Implement PollRange API endpoint 2023-01-10 15:22:25 +01:00
Alex Auvolat
43fd6c1526
PollRange RPC 2023-01-10 12:54:24 +01:00
Alex Auvolat
789540ca37
Type definition for range seen marker 2023-01-10 11:59:57 +01:00
Jonathan Davies
4cfb469d2b block/metrics.rs: Added compression_level metric. 2023-01-10 10:40:03 +00:00
Jonathan Davies
df1d9a9873 system.rs: Integrated SystemMetrics into System implementation. 2023-01-10 10:39:50 +00:00
Jonathan Davies
aac348fe93 Added system_metrics.rs file. 2023-01-10 10:38:50 +00:00
Alex Auvolat
9f5419f465
Make K2V item timestamps globally increasing on each node 2023-01-10 11:03:52 +01:00
Alex Auvolat
a48e2e0cb2
K2V: Subscription to ranges of items 2023-01-10 10:30:59 +01:00
Felix Scheinost
d6ea0cbefa Add tests for rpc_secret_file 2023-01-07 14:19:36 +01:00
Felix Scheinost
7b62fe3f0b Error on both rpc_secret and rpc_secret_file 2023-01-07 13:49:03 +01:00
Jonathan Davies
cb07e6145c Changed all instances of assignation to assignment. 2023-01-05 11:09:25 +00:00
Felix Scheinost
f2106c2733 Implement rpc_secret_file 2023-01-04 18:35:10 +01:00
Alex
02e8eb167e Merge pull request 'PutObject: better cleanup when request is interrupted in the middle' (#462) from interrupted-cleanup into main 
Reviewed-on: Deuxfleurs/garage#462
 2023-01-04 14:43:45 +00:00
Alex Auvolat
29dbcb8278
bg var operation on all nodes at once 2023-01-04 13:25:57 +01:00
Alex Auvolat
f3f27293df
Uniform framework for bg variable management 2023-01-04 13:07:13 +01:00
Alex Auvolat
13c5549886
Remove token_bucket.rs 2023-01-04 11:47:56 +01:00
Alex
80e4abb98d Merge pull request 'Changed all instances of 'key new' to 'key create' to make it the same as the bucket commands.' (#459) from jpds/garage:key-create-standardize into next 
Reviewed-on: Deuxfleurs/garage#459
 2023-01-04 10:35:49 +00:00
Alex Auvolat
570e5e5bbb
Merge branch 'main' into next 2023-01-04 11:34:43 +01:00
Alex Auvolat
936b6cb563
When saving block, delete .tmp file if we could not complete 2023-01-03 17:34:26 +01:00
Alex Auvolat
0650a43cf1
PutObject: better cleanup on Drop (incl. when request is interrupted in the middle) 2023-01-03 17:05:17 +01:00
Alex Auvolat
1fc220886a
Fix Consul & Kubernetes discovery with new way of doing background things 2023-01-03 16:55:59 +01:00
Alex Auvolat
1d5bdc17a4
use impossible enum type 2023-01-03 16:04:06 +01:00
Alex Auvolat
c106304b9c
more idiomatic and shorter 2023-01-03 16:00:19 +01:00
Alex Auvolat
33f25d26c7
fix doc and add tests for migrate.rs 2023-01-03 15:53:13 +01:00
Alex Auvolat
d6d571d512
cargo fmt 2023-01-03 15:30:21 +01:00
Alex Auvolat
a54b67740d
move debug_serialize to garage_util::encode 2023-01-03 15:29:29 +01:00
Alex Auvolat
8d5505514f
Make it explicit when using nonversioned encoding 2023-01-03 15:27:36 +01:00
Alex Auvolat
426d8784da
cleanup 2023-01-03 15:08:37 +01:00
Alex Auvolat
cdb2a591e9
Refactor how things are migrated 2023-01-03 14:44:47 +01:00
Jonathan Davies
8be862aa19 Changed all instances of 'key new' to 'key create' to make it consistent as bucket commands issued normally around the same time. 2023-01-03 11:11:12 +00:00
Alex Auvolat
939a6d67e8
Merge branch 'main' into internals-rework 2023-01-02 15:07:44 +01:00
Alex Auvolat
6775569525
Bump everything to v0.8.1 2023-01-02 14:15:33 +01:00
Alex Auvolat
6b857a9b8c
cargo fmt 2023-01-02 13:50:42 +01:00
Alex
7f7d53cfa9 Merge pull request 'improvements to CLI and new debug features' (#448) from cli-improvements into main 
Reviewed-on: Deuxfleurs/garage#448
 2023-01-02 12:42:24 +00:00
Alex
1af4a5ed56 Merge pull request 'Fix router keyword handling (fix #442)' (#446) from router-keywords-fix into main 
Reviewed-on: Deuxfleurs/garage#446
 2022-12-15 08:40:26 +00:00
Alex Auvolat
1fcd0b371b
online repair workers: retry on error 2022-12-14 16:31:31 +01:00
Alex Auvolat
13c8662126
factorize 2022-12-14 16:16:55 +01:00
Alex Auvolat
e6f14ab5cf
better error message handling 2022-12-14 16:11:19 +01:00
Alex Auvolat
510b620108
Get rid of background::spawn 2022-12-14 16:08:05 +01:00
Alex Auvolat
dfc131850a
Simplified and more aggressive worker exit logic 2022-12-14 15:25:29 +01:00
Alex Auvolat
d4af27f920
Add missing notify 2022-12-14 13:54:21 +01:00
Alex Auvolat
a19bfef508
Improve error message on rpc connection failure 2022-12-14 12:57:33 +01:00
Alex Auvolat
d56c472712
Refactor background runner and get rid of job worker 2022-12-14 12:51:42 +01:00
Alex Auvolat
2183518edc
Spawn all background workers in a separate step 2022-12-14 12:28:07 +01:00
Alex Auvolat
83c8467e23
Proper queueing for delayed inserts, now backed to disk 2022-12-14 11:58:06 +01:00
Alex Auvolat
f8e528c15d
Small refactor of tables internals 2022-12-14 10:48:49 +01:00
Alex Auvolat
d1279e04f3
Fix error messages 2022-12-13 16:18:01 +01:00
Alex Auvolat
041b60ed1d
Add block.rc_size, table.size and table.merkle_tree_size metrics 2022-12-13 15:54:03 +01:00
Alex Auvolat
f8d5409894
cli: more info displayed on error in garage stats 2022-12-13 15:46:04 +01:00
Alex Auvolat
d6040e32a6
cli: prettier table in garage stats 2022-12-13 15:43:22 +01:00
Alex Auvolat
d7f90cabb0
Implement block retry-now and block purge 2022-12-13 15:02:42 +01:00
Alex Auvolat
687660b27f
Implement block list-errors and block info 2022-12-13 14:23:45 +01:00
Alex Auvolat
9d82196945
cli: new worker info command 2022-12-13 12:24:30 +01:00
Alex Auvolat
a51e8d94c6
cli: rename resync-n-workers into resync-worker-count 2022-12-13 11:44:11 +01:00
Alex Auvolat
de9d6cddf7
Prettier worker list table; remove useless CLI log messages 2022-12-12 17:17:05 +01:00
Alex Auvolat
0e61e3b6fb
Fix bucket creation tests to take permissions into account 2022-12-12 15:47:55 +01:00
Alex Auvolat
a0abf41762
Fix router keyword handling (fix #442) 2022-12-12 12:05:37 +01:00
Alex Auvolat
2ac75018a1
Properly enforce allow_create_bucket 2022-12-12 12:03:54 +01:00
Alex
6e44369cbc Merge pull request 'Optimal layout assignation algorithm' (#296) from optimal-layout into next 
Reviewed-on: Deuxfleurs/garage#296
 2022-12-11 17:41:53 +00:00
Alex Auvolat
2c2e65ad8b
Merge commit 'ec12d6c' into next 2022-12-11 18:41:15 +01:00
Alex Auvolat
9d83364ad9
itertools .unique() doesn't require sorted items 2022-12-11 18:30:02 +01:00
Alex Auvolat
533afcf4e1
simplify 2022-12-11 18:17:08 +01:00
Alex Auvolat
5ea5fd2130
Always return 200 OK on /v0/health, reinstate admin api doc as draft and complete it 2022-12-11 18:11:28 +01:00
Alex Auvolat
d7868c48a4
Separate /health (simple text answer) and /v0/health (full json answer, authenticated) 2022-12-05 15:38:32 +01:00
Alex Auvolat
280d1be7b1
Refactor health check and add ability to return it in json 2022-12-05 15:28:57 +01:00
Alex Auvolat
2065f011ca
Implement /health admin API endpoint to check node health 2022-12-05 14:59:15 +01:00
Alex Auvolat
26121bb619
Fix logs appearing twice 2022-11-29 22:23:27 +01:00
Alex Auvolat
54e800ef8d
Tentative fix for issue #414 2022-11-21 17:13:41 +01:00
Alex Auvolat
c1fb65194c
Add sled default in garage_model also 2022-11-21 14:25:54 +01:00
Alex Auvolat
67941000ee
put sled as default feature in garage_db 2022-11-21 14:08:21 +01:00
Quentin Dufour
dc50fa3b34
Fix typo in admin API on BucketInfo 2022-11-11 16:56:56 +01:00
Alex Auvolat
ec12d6c8dd
Slightly simplify code at places 2022-11-08 16:15:45 +01:00
Alex Auvolat
217abdca18
Fix HTTP return code 2022-11-08 15:38:53 +01:00
Alex Auvolat
fc2729cd81
Fix integration test 2022-11-08 15:19:46 +01:00
Alex Auvolat
d75b37b018
Return more info when layout's .check() fails, fix compilation, fix test 2022-11-08 14:58:39 +01:00
Alex Auvolat
73a4ca8b15
Use bytes as capacity units 2022-11-07 21:12:11 +01:00
Alex Auvolat
fd5bc142b5
Ensure .sort() is called before counting unique items 2022-11-07 20:29:25 +01:00
Alex Auvolat
ea5afc2511
Style improvements 2022-11-07 20:11:30 +01:00
Alex Auvolat
28d7a49f63
Merge branch 'main' into optimal-layout 2022-11-07 12:20:59 +01:00
Alex Auvolat
e03d9062f7
Show a nice message and a backtrace when Garage panics 2022-11-04 16:39:02 +01:00
Alex Auvolat
8d3bbf5703
Clearer error messsages 2022-11-04 16:07:33 +01:00
Alex Auvolat
5b18fd8201
Add garage bucket cleanup-incomplete-uploads command 2022-11-04 11:55:59 +01:00
Alex Auvolat
57b5c2c754
Change reqwest rustls features 2022-10-18 22:11:27 +02:00
Alex Auvolat
8bc5caf7aa
Fix issue with 'http(s)://' prefix 2022-10-18 21:17:11 +02:00
Alex Auvolat
2da8786f54
move things around 2022-10-18 19:13:52 +02:00
Alex Auvolat
5d8d393054
Load TLS certificates only once 2022-10-18 19:11:16 +02:00
Alex Auvolat
002b9fc50c
Add TLS support for Consul discovery + refactoring 2022-10-18 18:38:20 +02:00
Alex
5670599372 Merge pull request 'Use status code 204 No Content for empty responses' (#403) from tobikris/garage:http-no-content into main 
Reviewed-on: Deuxfleurs/garage#403
 2022-10-18 14:20:44 +00:00
Tobias Krischer
7865003323
Use status code 204 No Content for empty responses 2022-10-17 10:55:26 +02:00
Alex Auvolat
c050a59fd0
Fix conditional testing in garage_db 2022-10-14 18:27:18 +02:00
Alex Auvolat
fcaee3bea0
definitively expunge openssl from dependencies everywhere 2022-10-14 18:10:36 +02:00
Alex Auvolat
8d04ae7014
cargo2nix unstable (patched), rust 1.63.0, nixpkgs 22.05 (32-bit builds are broken) 2022-10-14 14:30:48 +02:00
Mendes
bcdd1e0c33 Added some comment 2022-10-11 18:29:21 +02:00
Mendes
e5664c9822 Improved the statistics displayed in layout show 
corrected a few bugs
 2022-10-11 17:17:13 +02:00
Mendes
4abab246f1 cargo fmt 2022-10-10 17:21:13 +02:00
Mendes
fcf9ac674a Tests written in layout.rs 
added staged_parameters to ClusterLayout
removed the serde(default) -> will need a migration function
 2022-10-10 17:19:25 +02:00
Mendes
911eb17bd9 corrected warnings of cargo clippy 2022-10-06 14:53:57 +02:00
Mendes
9407df60cc Corrected two bugs: 
- self.node_id_vec was not properly updated when the previous ring was empty
- ClusterLayout::merge was not considering changes in the layout parameters
 2022-10-06 12:54:51 +02:00
Mendes
a951b6c452 Added a CLI command to update the parameters for the layout computation (for now, only the zone redundancy) 2022-10-05 16:04:19 +02:00
Mendes
ceac3713d6 modifications in several files to : 
- have consistent error return types
- store the zone redundancy in a Lww
- print the error and message in the CLI (TODO: for the server Api, should msg be returned in the body response?)
 2022-10-05 15:29:48 +02:00
Mendes
829f815a89 Merge remote-tracking branch 'origin/main' into optimal-layout 2022-10-04 18:14:49 +02:00
Mendes
99f96b9564 deleted zone_redundancy from System struct 2022-10-04 18:09:24 +02:00
Alex Auvolat
ad917ffd3f
Fix instant substractions that might have panicked 2022-09-29 15:53:54 +02:00
Alex Auvolat
1f97ce37e6
Shutdown properly on SIGTERM/SIGHUP and on Windows signals 2022-09-28 10:41:59 +02:00
Alex Auvolat
1778e4b318
Fix span name for api server requests 2022-09-26 16:21:30 +02:00
Mendes
bd842e1388 Correction of a few bugs in the tests, modification of ClusterLayout::check 2022-09-22 19:30:01 +02:00