Commit graph

71 commits

Author SHA1 Message Date
2e229d4430 new layout: improve output display 2023-09-12 17:24:51 +02:00
51abbb02d8 Merge branch 'main' into next 2023-09-11 20:00:02 +02:00
d5bb50d738 use statvfs instead of mount list to determine free data/meta space (fix #611) 2023-09-11 19:08:24 +02:00
ad6b1cc0be Merge branch 'main' into next 2023-09-11 13:14:18 +02:00
51eac97260 update version to 0.8.4 2023-09-05 23:28:12 +02:00
e78566591b Revert netapp update, hold to version 0.5.2 that uses rmp-serde 0.15 2023-09-05 23:23:23 +02:00
2e90e1c124 Merge branch 'main' into next 2023-08-29 11:32:42 +02:00
06369c8f4a add garage_db dependency in garage_rpc 2023-08-28 17:08:21 +02:00
cece1be1bb bump version to 0.8.3 2023-08-28 13:17:26 +02:00
Jonathan Davies
d66d81ae2d cargo: Updated gethostname v0.2.3 -> v0.4.3. 2023-08-28 09:30:27 +00:00
cda957b4b1 update netapp's rmp-serde dependency to v1.1 2023-06-13 17:34:49 +02:00
351d734e6c Merge branch 'main' into next 2023-05-09 12:40:08 +02:00
d2deee0b8b Declare garage crates using workspace.dependencies
This will allow to really disable "sled" feature without declaring
`default-features = false` in every Cargo.toml where garage_db and
garage_model is used.

See https://doc.rust-lang.org/cargo/reference/workspaces.html#the-dependencies-table
2023-05-09 08:46:15 +00:00
fa78d806e3 Merge branch 'main' into next 2023-04-25 12:34:26 +02:00
0a1ddcf630 Prepare for v0.8.2 2023-03-13 18:46:31 +01:00
teutat3s
f0717dd169
Bump pnet_datalink 0.28 -> 0.33
Motivation: building garage on illumos is only possible since
pnet_datalink version 0.30

Changelog: https://github.com/libpnet/libpnet/compare/v0.28.0...v0.33.0
2023-02-28 16:06:43 +01:00
94d559ae00 Merge branch 'main' into report-disk-usage 2023-01-26 16:20:41 +01:00
efb6b6e868 Disk space report
Report available disk space on nodes and calculate cluster-wide available space in `garage stats` (fix #479)
2023-01-26 15:04:32 +01:00
Jonathan Davies
20c1cdf662 Cargo.toml: Loosen tracing dependency to just 0.1. 2023-01-26 11:13:11 +00:00
570e5e5bbb
Merge branch 'main' into next 2023-01-04 11:34:43 +01:00
8d5505514f
Make it explicit when using nonversioned encoding 2023-01-03 15:27:36 +01:00
6775569525
Bump everything to v0.8.1 2023-01-02 14:15:33 +01:00
73a4ca8b15
Use bytes as capacity units 2022-11-07 21:12:11 +01:00
28d7a49f63
Merge branch 'main' into optimal-layout 2022-11-07 12:20:59 +01:00
57b5c2c754
Change reqwest rustls features 2022-10-18 22:11:27 +02:00
002b9fc50c
Add TLS support for Consul discovery + refactoring 2022-10-18 18:38:20 +02:00
fcaee3bea0
definitively expunge openssl from dependencies everywhere 2022-10-14 18:10:36 +02:00
829f815a89 Merge remote-tracking branch 'origin/main' into optimal-layout 2022-10-04 18:14:49 +02:00
56592e1853
RPC performance changes
- configurable ping timeout
- single, much higher, configurable RPC timeout
- no more concurrency semaphore
2022-09-19 20:31:00 +02:00
e46dc2a8ef
Allow for hostnames in bootstrap_peers and rpc_public_addr (fix #353) 2022-09-14 16:09:38 +02:00
28a4af73ca
Use netapp 0.5 published from crates.io 2022-09-13 13:11:44 +02:00
7f54706b95
Merge branch 'lx-perf-improvements' into netapp-stream-body 2022-09-08 15:50:56 +02:00
d9d199a6c9
Merge branch 'main' into lx-perf-improvements 2022-09-08 15:49:17 +02:00
6b958979bd
Merge branch 'lx-perf-improvements' into netapp-stream-body 2022-09-06 22:13:01 +02:00
48ffaaadfc
Bump versions to 0.8.0 (compatibility is broken already) 2022-09-06 16:47:56 +02:00
a6e40b75ea Add feature "system-libs" to enable linking against system libraries
If this feature is enabled, libsodium-sys and zstd-sys will link
dynamically against system-provided libraries instead of building
and linking statically the bundled (possibly outdated and vulnerable)
copies of them. This feature is intended mainly for linux package
maintainers.
2022-09-03 18:44:34 +02:00
6226f5ceca
Update to netapp 0.4.5 - fixed ping 2022-09-02 14:33:12 +02:00
1921f4f7e6
Merge branch 'lx-perf-improvements' into netapp-stream-body 2022-08-29 16:45:05 +02:00
2c7bae935a
Configure structopt to report the right version
By default, structopt reports the value provided by
the env var CARGO_PKG_VERSION, feeded by Cargo when reading
Cargo.toml. However for Garage we use a versioning based on git,
so we often report a version that is behind the real version.
In this commit, we create garage_util::version::garage() that
reports the right version and configure all structopt subcommands
to call this function instead of using the env var.
2022-08-11 10:21:45 +02:00
a35d4da721
update netapp to 0.5 2022-07-29 12:25:02 +02:00
8e7e680afe
First adaptation to WIP netapp with streaming body 2022-07-29 12:25:02 +02:00
382e74c798 First version of admin API (#298)
**Spec:**

- [x] Start writing
- [x] Specify all layout endpoints
- [x] Specify all endpoints for operations on keys
- [x] Specify all endpoints for operations on key/bucket permissions
- [x] Specify all endpoints for operations on buckets
- [x] Specify all endpoints for operations on bucket aliases

View rendered spec at <https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/admin-api/doc/drafts/admin-api.md>

**Code:**

- [x] Refactor code for admin api to use common api code that was created for K2V

**General endpoints:**

- [x] Metrics
- [x] GetClusterStatus
- [x] ConnectClusterNodes
- [x] GetClusterLayout
- [x] UpdateClusterLayout
- [x] ApplyClusterLayout
- [x] RevertClusterLayout

**Key-related endpoints:**

- [x] ListKeys
- [x] CreateKey
- [x] ImportKey
- [x] GetKeyInfo
- [x] UpdateKey
- [x] DeleteKey

**Bucket-related endpoints:**

- [x] ListBuckets
- [x] CreateBucket
- [x] GetBucketInfo
- [x] DeleteBucket
- [x] PutBucketWebsite
- [x] DeleteBucketWebsite

**Operations on key/bucket permissions:**

- [x] BucketAllowKey
- [x] BucketDenyKey

**Operations on bucket aliases:**

- [x] GlobalAliasBucket
- [x] GlobalUnaliasBucket
- [x] LocalAliasBucket
- [x] LocalUnaliasBucket

**And also:**

- [x] Separate error type for the admin API (this PR includes a quite big refactoring of error handling)
- [x] Add management of website access
- [ ] Check that nothing is missing wrt what can be done using the CLI
- [ ] Improve formatting of the spec
- [x] Make sure everyone is cool with the API design

Fix #231
Fix #295

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: Deuxfleurs/garage#298
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-05-24 12:16:39 +02:00
5768bf3622 First implementation of K2V (#293)
**Specification:**

View spec at [this URL](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/k2v/doc/drafts/k2v-spec.md)

- [x] Specify the structure of K2V triples
- [x] Specify the DVVS format used for causality detection
- [x] Specify the K2V index (just a counter of number of values per partition key)
- [x] Specify single-item endpoints: ReadItem, InsertItem, DeleteItem
- [x] Specify index endpoint: ReadIndex
- [x] Specify multi-item endpoints: InsertBatch, ReadBatch, DeleteBatch
- [x] Move to JSON objects instead of tuples
- [x] Specify endpoints for polling for updates on single values (PollItem)

**Implementation:**

- [x] Table for K2V items, causal contexts
- [x] Indexing mechanism and table for K2V index
- [x] Make API handlers a bit more generic
- [x] K2V API endpoint
- [x] K2V API router
- [x] ReadItem
- [x] InsertItem
- [x] DeleteItem
- [x] PollItem
- [x] ReadIndex
- [x] InsertBatch
- [x] ReadBatch
- [x] DeleteBatch

**Testing:**

- [x] Just a simple Python script that does some requests to check visually that things are going right (does not contain parsing of results or assertions on returned values)
- [x] Actual tests:
  - [x] Adapt testing framework
  - [x] Simple test with InsertItem + ReadItem
  - [x] Test with several Insert/Read/DeleteItem + ReadIndex
  - [x] Test all combinations of return formats for ReadItem
  - [x] Test with ReadBatch, InsertBatch, DeleteBatch
  - [x] Test with PollItem
  - [x] Test error codes
- [ ] Fix most broken stuff
  - [x] test PollItem broken randomly
  - [x] when invalid causality tokens are given, errors should be 4xx not 5xx

**Improvements:**

- [x] Descending range queries
  - [x] Specify
  - [x] Implement
  - [x] Add test
- [x] Batch updates to index counter
- [x] Put K2V behind `k2v` feature flag

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: Deuxfleurs/garage#293
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-05-10 13:16:57 +02:00
def78c5e6f
Update netapp to 0.4.4, fix #300 2022-05-09 12:08:47 +02:00
c1d1646c4d
Change the way new layout assignations are computed.
The function now computes an optimal assignation (with respect to partition size) that minimizes the distance to the former assignation, using flow algorithms.

This commit was written by Mendes Oulamara <mendes.oulamara@pm.me>
2022-05-01 09:54:19 +02:00
94f1e48fff Update to netapp 0.4.2 (a tiny fix) 2022-04-07 11:50:03 +02:00
9d0ed78887 Add feature flag for Kubernetes discovery 2022-03-24 16:57:43 +01:00
db46cdef79
Update netapp to v0.4.1 2022-03-15 17:09:57 +01:00
203e8d2c34
Bump version to 0.7 because of incompatible Netapp 2022-03-14 10:54:24 +01:00
0cc31ee169
add missing netapp telemetry feature 2022-03-14 10:54:11 +01:00