Commit graph

1835 commits

Author SHA1 Message Date
1e92e9f782
Disable k2v tests when feature is disabled
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-09-06 17:29:46 +02:00
431dee050f
Remove opentelemetry-otlp dep in api/
Some checks failed
continuous-integration/drone/push Build is failing
continuous-integration/drone/pr Build is failing
2022-09-06 17:25:44 +02:00
2c2b93acdf
Update Nix files with optional db engines
Some checks are pending
continuous-integration/drone/push Build is pending
continuous-integration/drone/pr Build is pending
2022-09-06 17:20:10 +02:00
bbb970965c
Document available build features
Some checks are pending
continuous-integration/drone/push Build is pending
continuous-integration/drone/pr Build is pending
2022-09-06 17:16:45 +02:00
b886c75450
Make all DB engines optional build features 2022-09-06 17:09:43 +02:00
48ffaaadfc
Bump versions to 0.8.0 (compatibility is broken already)
Some checks failed
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2022-09-06 16:47:56 +02:00
7de53a4d66
Force disable pkg-config for libsodum-sys and libzstd-sys
Some checks failed
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2022-09-06 16:41:58 +02:00
8d77a76df1
Update .nix files
Some checks failed
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2022-09-06 15:49:41 +02:00
454d8474ef
Fix clippy 2022-09-06 15:43:50 +02:00
ed7796924b Merge pull request 'Make OTLP exporter optional and allow building without Prometheus exporter (/metrics)' (#372) from jirutka/garage:telemetry-and-metrics into improve-deps
Some checks failed
continuous-integration/drone/push Build is failing
continuous-integration/drone/pr Build is failing
Reviewed-on: #372
Reviewed-by: Alex <alex@adnab.me>
2022-09-06 15:11:30 +02:00
ea36b9ff90 Allow building without Prometheus exporter (/metrics endpoint)
Some checks failed
continuous-integration/drone/pr Build is failing
prometheus and opentelemetry-prometheus add 7 extra dependencies in
total and increases the size of the garage binary by ~7 % (with
fat LTO).
2022-09-06 01:15:09 +02:00
e7af006c1c Make OTLP exporter optional via feature "telemetry-otlp"
opentelemetry-otlp add 48 (!) extra dependencies and increases the
size of the garage binary by ~11 % (with fat LTO).
2022-09-06 01:14:47 +02:00
db72812f01 Use the new cargo feature resolver "2"
Garage currently uses the legacy resolver "1". The new one is used
by default if the root package specifies 'edition = 2021', which
Garage does not (yet).

The problem with the legacy resolver is, among others, that features
enabled by dev-dependencies are propagated to normal dependencies.
This affects e.g. hyper - one of the dev-dependencies enables "http2"
feature that adds many extra dependencies. If we build garage without
opentelemetry-otlp (this is enabled in the following commit), there's
no normal dependency enabling "http2" feature.

See https://doc.rust-lang.org/cargo/reference/resolver.html#feature-resolver-version-2
2022-09-06 01:14:19 +02:00
729a910e14
Remove Heed default features
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-09-05 16:40:13 +02:00
9f5433db82 Merge pull request 'Update .drone.yml signature' (#374) from fix-drone-signature into main
All checks were successful
continuous-integration/drone/push Build is passing
Reviewed-on: #374
2022-09-05 16:18:15 +02:00
fd8074ad9b
Update .drone.yml signature
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2022-09-05 16:09:01 +02:00
07e6bcde85
Merge branch 'main' into lx-perf-improvements
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2022-09-05 12:40:17 +02:00
0009fd136c Merge pull request 'Make block resync speed dynamically configurable' (#369) from resync-ajustable-speed into main
All checks were successful
continuous-integration/drone/push Build is passing
Included in this PR:

- [x] Small refactor, resync code is moved to a separate `block/resync.rs` file
- [x] Block resync tranquility is no longer in config file, it is set dynamically using `garage worker set resync-tranquility` (this parameter is persisted over Garage restarts)
- [x] Up to 4 block resync workers can be activated to run simultaneously to speed up big resyncs, this parameter is set dynamically using `garage worker set resync-n-workers`

Reviewed-on: #369
2022-09-05 12:35:08 +02:00
7511ba5530 Allow linking against system-provided libsqlite
Some checks are pending
continuous-integration/drone/push Build is pending
continuous-integration/drone/pr Build is pending
Unfortunately, rusqlite uses the opposite logic for enabling/disabling
bundled libraries to others (libsodium-sys, zstd-sys). Cargo features
are very limited and doesn't allow to enable feature A in a dependency
iff feature B is disabled.

Note, lmdb-rkv-sys doesn't need any special treatment because it
automatically links against system liblmdb if found via pkgconf.

Linux distros should build garage with
`--no-default-features --features system-libs` to disable bundled-libs
and enable system-libs.
2022-09-03 19:15:57 +02:00
a6e40b75ea Add feature "system-libs" to enable linking against system libraries
If this feature is enabled, libsodium-sys and zstd-sys will link
dynamically against system-provided libraries instead of building
and linking statically the bundled (possibly outdated and vulnerable)
copies of them. This feature is intended mainly for linux package
maintainers.
2022-09-03 18:44:34 +02:00
e1751c8a9c
fix clippy
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-09-02 17:24:26 +02:00
5d4b937a00
Ability to have up to 4 concurrently working resync workers
Some checks failed
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2022-09-02 17:18:13 +02:00
5e8baa433d
Make BlockManagerLocked fully private again
All checks were successful
continuous-integration/drone/push Build is passing
2022-09-02 16:52:22 +02:00
47be652a1f
block manager: refactor: split resync into separate file
All checks were successful
continuous-integration/drone/push Build is passing
2022-09-02 16:47:15 +02:00
943d76c583
Ability to dynamically set resync tranquility
All checks were successful
continuous-integration/drone/push Build is passing
2022-09-02 15:34:21 +02:00
6226f5ceca
Update to netapp 0.4.5 - fixed ping
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-09-02 14:33:12 +02:00
13b5f28c7e
Make use of BytesBuf from new Netapp
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-09-02 13:46:42 +02:00
1ef87ac4cb
cargo fmt
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2022-09-02 13:38:29 +02:00
99b532b85b
Apply PRIO_SECONDARY to block data transfers
Some checks failed
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2022-09-01 16:35:43 +02:00
e648bf7b69
update cargo.nix
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2022-09-01 16:31:04 +02:00
df094bd807
Less strict timeouts 2022-09-01 16:30:44 +02:00
f3bf34b6a1
update netapp: straming + fix-ping
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-09-01 14:23:54 +02:00
bc977f9a7a
Update to Netapp with OrderTag support and exploit OrderTags
Some checks failed
continuous-integration/drone/push Build is failing
continuous-integration/drone/pr Build is failing
2022-09-01 12:58:20 +02:00
4b726b0941
netapp recv with unbounded channel removes deadlock
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-09-01 09:47:28 +02:00
70231d68b2
Fix bytes_read counter
Some checks failed
continuous-integration/drone/push Build is failing
continuous-integration/drone/pr Build is failing
2022-08-31 19:44:27 +02:00
e598231ca4
update netapp git commit
Some checks failed
continuous-integration/drone/push Build is failing
continuous-integration/drone/pr Build is failing
2022-08-31 19:27:25 +02:00
c9bc9d89de
Merge branch 'lx-perf-improvements' into netapp-stream-body
Some checks failed
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2022-08-31 17:42:31 +02:00
eb97e13a6a
update cargo.nix
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-08-31 17:42:00 +02:00
efbca67ce4
Add env filter to tracing subscriber
Some checks failed
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2022-08-31 14:39:12 +02:00
44cd98d2e4
Tracing-subscriber: write to stderr
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-08-31 14:28:17 +02:00
dd5304f6fc
Replace logging crate pretty_env_logger by tracing_subscriber::fmt
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2022-08-31 14:24:41 +02:00
322dafc761
Try to fix clippy
Some checks reported errors
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build was killed
continuous-integration/drone Build was killed
2022-08-29 17:32:45 +02:00
5d065b8a0f
cargo2nix fix to fetchCrateGit
Some checks failed
continuous-integration/drone/push Build is failing
continuous-integration/drone/pr Build is failing
2022-08-29 17:24:53 +02:00
52749e28f7
Merge branch 'lx-perf-improvements' into netapp-stream-body
Some checks failed
continuous-integration/drone/push Build is failing
continuous-integration/drone/pr Build is failing
continuous-integration/drone Build is failing
2022-08-29 16:48:43 +02:00
4da67b0035
Update drone signature
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
continuous-integration/drone Build is passing
2022-08-29 16:48:31 +02:00
1921f4f7e6
Merge branch 'lx-perf-improvements' into netapp-stream-body
Some checks failed
continuous-integration/drone/push Build is failing
continuous-integration/drone/pr Build is failing
2022-08-29 16:45:05 +02:00
ebc20a8798
Merge branch 'main' into lx-perf-improvements
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2022-08-29 16:44:13 +02:00
81083dd415 Added a first draft version of the algorithm and analysis for the non-strict mode.
Some checks reported errors
continuous-integration/drone/push Build was killed
continuous-integration/drone/pr Build was killed
2022-08-19 21:21:41 +02:00
532eca7ff9
Add some documentation for Caddy
Some checks are pending
continuous-integration/drone/pr Build is passing
continuous-integration/drone Build is passing
continuous-integration/drone/push Build is pending
2022-08-12 10:33:41 +02:00
2c7bae935a
Configure structopt to report the right version
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/tag Build is passing
continuous-integration/drone Build is passing
continuous-integration/drone/push Build is passing
By default, structopt reports the value provided by
the env var CARGO_PKG_VERSION, feeded by Cargo when reading
Cargo.toml. However for Garage we use a versioning based on git,
so we often report a version that is behind the real version.
In this commit, we create garage_util::version::garage() that
reports the right version and configure all structopt subcommands
to call this function instead of using the env var.
2022-08-11 10:21:45 +02:00