rune/garage
1
0
Fork 0
forked from Deuxfleurs/garage
Code Issues Pull requests Projects Releases Wiki Activity
2084 commits 14 branches 0 tags 70 MiB
Commit graph

1206 commits

Author SHA1 Message Date
Alex Auvolat
390a5d97fe nix, ci: build with Crane 
This removes our dependency on cargo2nix, which was causing us some
issues. Whereas cargo2nix creates one Nix derivation per crate, Crane
uses only two derivations:

1. Build dependencies only
2. Build the final binary

This means that during the second step, no caching can be done. For
instance, if we do a change in garage_model, we need to recompile all of
the Garage crates including those that do not depend on garage_model.
On the upside, this allows all of the Garage crates to be built at once
using cargo build logic, which is optimized for high parallelism and
better pipelining between all of the steps of the build. All in all,
this makes most builds faster than cargo2nix.

A few other changes have been made to the build scripts and CI:

- Unit tests are now run within a Nix derivation. In fact, we have
  different derivations to run the tests using LMDB and Sqlite as
  metadata db engines.

- For debug builds, most CI steps now run in parallel (with the notable
  exception of the smoke test that runs after the build, which is
  inevitable).

- We no longer pass the GIT_VERSION argument when building debug builds
  and running the tests. This means that dev binaries and test
  binaries don't know the exact version of Garage they are from. That
  shouldn't be an issue in most cases.

- The not-dynamic.sh scripts has been fixed to fail if the file does not
  exist.
 2025-02-03 16:39:50 +01:00
Alex Auvolat
d18c5ad0ff fix tests 2025-01-31 19:12:51 +01:00
Alex Auvolat
3d5e9a027e cargo defs: simplify and fix descriptions 2025-01-31 18:54:29 +01:00
Alex Auvolat
4563313f87 use cargo-shear to remove many unused dependencies between crates 2025-01-31 18:47:30 +01:00
Alex Auvolat
afa28706e5 split s3/cors.rs into also common/cors.rs 2025-01-31 18:42:14 +01:00
Alex Auvolat
84f1db91c4 fix things up 2025-01-31 18:34:57 +01:00
Alex Auvolat
9fa20d45be wip: split garage_api into garage_api_{common,s3,k2v,admin} 2025-01-31 18:18:29 +01:00
Alex Auvolat
83f6928ff7 table::insert_many: avoid failure with zero items (fix #915) 2025-01-30 18:06:47 +01:00
Alex Auvolat
9f3c7c3720 api: better handling of helper errors to distinguish error codes 2025-01-29 19:14:34 +01:00
Alex
a1d081ee84 Merge pull request 's3 api: make x-amz-meta-* headers lowercase (fix #844)' (#938) from fix-844 into main 
Reviewed-on: Deuxfleurs/garage#938
 2025-01-27 19:32:19 +00:00
Alex Auvolat
e8fa89e834 s3 api: make x-amz-meta-* headers lowercase (fix #844) 2025-01-27 19:58:06 +01:00
Baptiste Jonglez
43402c9619 snapshot: sqlite: use a subdirectory for consistency with LMDB 
Currently, taking a snapshot of the metadata database with sqlite creates
a sqlite file without extension with the following format:

    snapshots/2025-01-26T15:29:17Z

This makes it hard to understand what kind of data this is, and is not
consistent with LMDB:

    snapshots/2025-01-26T15:29:17Z/data.mdb

With this change, we now get a directory with a single db.sqlite file:

    snapshots/2025-01-26T15:29:17Z/db.sqlite
 2025-01-27 19:06:52 +01:00
Baptiste Jonglez
59c153d280 db-snapshot: allow to set directory where snapshots are stored 
Fix #926
 2025-01-27 18:33:55 +01:00
Stefan Majer
2eb9fcae20 Fix all typos 2025-01-16 13:22:00 +01:00
Alex Auvolat
295237476e fix formatting to comply with latest rustfmt 2025-01-12 17:36:25 +01:00
Vedad KAJTAZ
6689800986 Formatting with 2025-01-04 16:52:23 +01:00
Vedad KAJTAZ
6ca99fd02c formatting 2025-01-04 14:46:42 +01:00
Vedad KAJTAZ
b568bb863d Fix #907 2025-01-04 12:50:10 +01:00
Renjaya Raga Zenta
4c1bf42192
feat: add use_local_tz configuration 
Used in lifecycle_worker to determine midnight time
 2024-11-23 05:51:12 +07:00
trinity-1686a
9bd9e392ba fix bit/byte inversion in rpc secret error message 2024-11-07 00:29:26 +01:00
Alex Auvolat
7a143f46fc
Bump to version 1.0.1 2024-09-22 14:25:32 +02:00
Alex Auvolat
34453bc9c2
fix logic in garage layout skip-dead-nodes + fix typo (fix #879) 2024-09-22 13:47:27 +02:00
Julien Kritter
bd71728874
Tests: don't expect old value after transactional insert 2024-09-12 10:50:53 +02:00
Julien Kritter
51ced60366
Don't fetch old values in cross-partition transactional inserts 2024-09-12 10:26:28 +02:00
marvin-j97
8062ec7b4b test: fix db tests 2024-09-04 19:24:36 +02:00
marvin-j97
eb416a02fb dont assert deletion count in sqlite KV adapter 2024-09-04 18:51:51 +02:00
marvin-j97
74363c9060 perf(kv): dont retrieve values for write ops 
see Deuxfleurs/garage#851
 2024-09-04 18:45:17 +02:00
Alex Auvolat
baf32c9575
api servers: kill opened connections after SIGINT after 10s deadline (fix #806) 2024-08-25 20:04:56 +02:00
Alex Auvolat
aa7ce9e97c
fix build when lmdb feature is disabled (fix #800) 2024-08-25 11:42:37 +02:00
Alex
8d62616ec0 Merge pull request 'layout: discard old info when it is completely out-of-date (fix #841)' (#861) from fix-841 into main 
Reviewed-on: Deuxfleurs/garage#861
 2024-08-24 11:12:39 +00:00
Alex
3e711bc110 Merge pull request 'don't modify postobject request before validating policy' (#850) from trinity-1686a/garage:fix-acl-postobject into main 
Reviewed-on: Deuxfleurs/garage#850
 2024-08-24 10:49:14 +00:00
Alex Auvolat
7fb66b4944
layout: discard old info when it is completely out-of-date (fix #841) 2024-08-24 12:38:56 +02:00
Quentin Dufour
2a93ad0c84
force flag "no read ahead" on LMDB 2024-08-17 21:17:15 +02:00
trinity-1686a
f190032589 don't modify postobject request before validating policy 2024-08-10 20:10:47 +02:00
Maximilien R.
9302cd42f0 Improve error message for malformed RPC secret key 2024-08-08 23:05:24 +00:00
Florian Klink
a0f6bc5b7f add rpc_public_addr_subnet config option 
In case `rpc_public_addr` is not set, but autodiscovery is used, this
allows filtering the list of automatically discovered IPs to a specific
subnet.

For example, if nodes should pick *their* IP inside a specific subnet,
but you don't want to explicitly write the IP down (as it's dynamic, or
you want to share configs across nodes), you can use this option.
 2024-06-05 08:41:36 +02:00
Florian Klink
15847a636a cli: clarify garage block is node-local 
Prevents some of the confusion from
Deuxfleurs/garage#810.
 2024-05-07 07:42:33 +00:00
Alex Auvolat
90e3c2af91
[next-0.10] small updates to mention Garage v0.9.4 2024-04-10 14:35:30 +02:00
Alex Auvolat
afad62939e
[next-0.10] bump version number to 1.0 2024-03-28 15:19:44 +01:00
Alex Auvolat
8bfc16ba7d
Merge branch 'main' into next-0.10 2024-03-28 15:01:05 +01:00
Alex Auvolat
85f580cbde
[fix-buffering] change request sending strategy and fix priorities 
remove LAS, priorize new requests but otherwise just do standard queuing
 2024-03-27 16:22:40 +01:00
Alex Auvolat
0d3e285d13
[fix-buffering] implement block_ram_buffer_max to avoid excessive RAM usage 2024-03-27 16:22:40 +01:00
Alex Auvolat
25c196f34d
[next-0.10] admin api: fix logic in get cluster status 2024-03-27 13:55:49 +01:00
Alex Auvolat
4eba32f29f
[next-0.10] layout helper: rename & clarify updates to update trackers 2024-03-27 13:47:06 +01:00
Alex Auvolat
32f1786f9f
[next-0.10] cache layout check result 2024-03-27 13:37:20 +01:00
Alex Auvolat
01a0bd5410
[next-0.10] remove impl Deref for LayoutHelper 2024-03-27 13:32:13 +01:00
Alex Auvolat
c0eeb0b0f3
[next-0.10] fixes to k2v rpc + comment fixes 2024-03-27 10:44:03 +01:00
Alex Auvolat
f7cd4eb600
Merge branch 'main' into next-0.10 2024-03-26 16:34:40 +01:00
Alex Auvolat
73551e9a2d
[disable-k2v-test] disable the other k2v poll test 2024-03-26 16:24:26 +01:00
Alex Auvolat
f267609343
[disable-k2v-test] disable tests::k2v::test_poll_item as it is not 100% reliable 2024-03-26 15:39:17 +01:00