Commit graph

77 commits

Author SHA1 Message Date
Alex 6a7623e90d
[rel-0.9.2] Bump version to v0.9.2
All checks were successful
ci/woodpecker/push/debug Pipeline was successful
ci/woodpecker/pr/debug Pipeline was successful
2024-03-01 16:54:39 +01:00
Alex 5ea24254a9
[import-netapp] import Netapp code into Garage codebase 2024-02-15 12:15:07 +01:00
Alex fe1af5d98b
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2024-02-05 13:02:02 +01:00
Alex ee57dd922b
Bump version to 0.9.1
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2024-01-16 16:28:17 +01:00
Alex 952c9570c4 bump version to v0.9.0
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
continuous-integration/drone Build is passing
2023-10-10 14:08:11 +02:00
Alex 897cbf2c27 actually update rmp-serde to 1.1.2 for both garage and netapp dependency (fix #629)
Some checks reported errors
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
continuous-integration/drone/tag Build is passing
continuous-integration/drone Build was killed
2023-09-27 13:13:00 +02:00
Alex 2e229d4430 new layout: improve output display
Some checks reported errors
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build was killed
2023-09-12 17:24:51 +02:00
Alex 51abbb02d8 Merge branch 'main' into next
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2023-09-11 20:00:02 +02:00
Alex d5bb50d738 use statvfs instead of mount list to determine free data/meta space (fix #611)
Some checks reported errors
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
continuous-integration/drone Build was killed
2023-09-11 19:08:24 +02:00
Alex ad6b1cc0be Merge branch 'main' into next
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2023-09-11 13:14:18 +02:00
Alex 51eac97260 update version to 0.8.4
Some checks failed
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
continuous-integration/drone Build is failing
2023-09-05 23:28:12 +02:00
Alex e78566591b Revert netapp update, hold to version 0.5.2 that uses rmp-serde 0.15
Some checks failed
continuous-integration/drone/push Build is failing
2023-09-05 23:23:23 +02:00
Alex 2e90e1c124 Merge branch 'main' into next
Some checks reported errors
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build was killed
continuous-integration/drone Build is passing
2023-08-29 11:32:42 +02:00
Alex 06369c8f4a add garage_db dependency in garage_rpc
Some checks reported errors
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
continuous-integration/drone Build was killed
2023-08-28 17:08:21 +02:00
Alex cece1be1bb bump version to 0.8.3
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
continuous-integration/drone Build is passing
2023-08-28 13:17:26 +02:00
Jonathan Davies d66d81ae2d cargo: Updated gethostname v0.2.3 -> v0.4.3.
Some checks failed
continuous-integration/drone/pr Build is failing
2023-08-28 09:30:27 +00:00
Alex cda957b4b1 update netapp's rmp-serde dependency to v1.1
Some checks failed
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2023-06-13 17:34:49 +02:00
Alex 351d734e6c Merge branch 'main' into next
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2023-05-09 12:40:08 +02:00
Jakub Jirutka d2deee0b8b Declare garage crates using workspace.dependencies
This will allow to really disable "sled" feature without declaring
`default-features = false` in every Cargo.toml where garage_db and
garage_model is used.

See https://doc.rust-lang.org/cargo/reference/workspaces.html#the-dependencies-table
2023-05-09 08:46:15 +00:00
Alex fa78d806e3 Merge branch 'main' into next 2023-04-25 12:34:26 +02:00
Alex 0a1ddcf630 Prepare for v0.8.2
Some checks failed
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2023-03-13 18:46:31 +01:00
teutat3s f0717dd169
Bump pnet_datalink 0.28 -> 0.33
Some checks failed
continuous-integration/drone/pr Build is failing
Motivation: building garage on illumos is only possible since
pnet_datalink version 0.30

Changelog: https://github.com/libpnet/libpnet/compare/v0.28.0...v0.33.0
2023-02-28 16:06:43 +01:00
Alex 94d559ae00 Merge branch 'main' into report-disk-usage
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2023-01-26 16:20:41 +01:00
Alex efb6b6e868 Disk space report
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
Report available disk space on nodes and calculate cluster-wide available space in `garage stats` (fix #479)
2023-01-26 15:04:32 +01:00
Jonathan Davies 20c1cdf662 Cargo.toml: Loosen tracing dependency to just 0.1. 2023-01-26 11:13:11 +00:00
Alex 570e5e5bbb
Merge branch 'main' into next
All checks were successful
continuous-integration/drone/push Build is passing
2023-01-04 11:34:43 +01:00
Alex 8d5505514f
Make it explicit when using nonversioned encoding
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2023-01-03 15:27:36 +01:00
Alex 6775569525
Bump everything to v0.8.1
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2023-01-02 14:15:33 +01:00
Alex 73a4ca8b15
Use bytes as capacity units
Some checks failed
continuous-integration/drone/push Build is failing
continuous-integration/drone/pr Build is failing
2022-11-07 21:12:11 +01:00
Alex 28d7a49f63
Merge branch 'main' into optimal-layout
Some checks failed
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2022-11-07 12:20:59 +01:00
Alex 57b5c2c754
Change reqwest rustls features
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-10-18 22:11:27 +02:00
Alex 002b9fc50c
Add TLS support for Consul discovery + refactoring
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-10-18 18:38:20 +02:00
Alex fcaee3bea0
definitively expunge openssl from dependencies everywhere
Some checks reported errors
continuous-integration/drone/push Build encountered an error
continuous-integration/drone/pr Build is failing
2022-10-14 18:10:36 +02:00
Mendes 829f815a89 Merge remote-tracking branch 'origin/main' into optimal-layout
Some checks failed
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2022-10-04 18:14:49 +02:00
Alex 56592e1853
RPC performance changes
Some checks reported errors
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
continuous-integration/drone Build was killed
- configurable ping timeout
- single, much higher, configurable RPC timeout
- no more concurrency semaphore
2022-09-19 20:31:00 +02:00
Alex e46dc2a8ef
Allow for hostnames in bootstrap_peers and rpc_public_addr (fix #353)
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-09-14 16:09:38 +02:00
Alex 28a4af73ca
Use netapp 0.5 published from crates.io
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2022-09-13 13:11:44 +02:00
Alex 7f54706b95
Merge branch 'lx-perf-improvements' into netapp-stream-body
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2022-09-08 15:50:56 +02:00
Alex d9d199a6c9
Merge branch 'main' into lx-perf-improvements
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2022-09-08 15:49:17 +02:00
Alex 6b958979bd
Merge branch 'lx-perf-improvements' into netapp-stream-body
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2022-09-06 22:13:01 +02:00
Alex 48ffaaadfc
Bump versions to 0.8.0 (compatibility is broken already)
Some checks failed
continuous-integration/drone/pr Build is failing
continuous-integration/drone/push Build is failing
2022-09-06 16:47:56 +02:00
Jakub Jirutka a6e40b75ea Add feature "system-libs" to enable linking against system libraries
If this feature is enabled, libsodium-sys and zstd-sys will link
dynamically against system-provided libraries instead of building
and linking statically the bundled (possibly outdated and vulnerable)
copies of them. This feature is intended mainly for linux package
maintainers.
2022-09-03 18:44:34 +02:00
Alex 6226f5ceca
Update to netapp 0.4.5 - fixed ping
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-09-02 14:33:12 +02:00
Alex 1921f4f7e6
Merge branch 'lx-perf-improvements' into netapp-stream-body
Some checks failed
continuous-integration/drone/push Build is failing
continuous-integration/drone/pr Build is failing
2022-08-29 16:45:05 +02:00
Quentin 2c7bae935a
Configure structopt to report the right version
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/tag Build is passing
continuous-integration/drone Build is passing
continuous-integration/drone/push Build is passing
By default, structopt reports the value provided by
the env var CARGO_PKG_VERSION, feeded by Cargo when reading
Cargo.toml. However for Garage we use a versioning based on git,
so we often report a version that is behind the real version.
In this commit, we create garage_util::version::garage() that
reports the right version and configure all structopt subcommands
to call this function instead of using the env var.
2022-08-11 10:21:45 +02:00
Alex a35d4da721
update netapp to 0.5 2022-07-29 12:25:02 +02:00
Alex 8e7e680afe
First adaptation to WIP netapp with streaming body 2022-07-29 12:25:02 +02:00
Alex 382e74c798 First version of admin API (#298)
All checks were successful
continuous-integration/drone/push Build is passing
**Spec:**

- [x] Start writing
- [x] Specify all layout endpoints
- [x] Specify all endpoints for operations on keys
- [x] Specify all endpoints for operations on key/bucket permissions
- [x] Specify all endpoints for operations on buckets
- [x] Specify all endpoints for operations on bucket aliases

View rendered spec at <https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/admin-api/doc/drafts/admin-api.md>

**Code:**

- [x] Refactor code for admin api to use common api code that was created for K2V

**General endpoints:**

- [x] Metrics
- [x] GetClusterStatus
- [x] ConnectClusterNodes
- [x] GetClusterLayout
- [x] UpdateClusterLayout
- [x] ApplyClusterLayout
- [x] RevertClusterLayout

**Key-related endpoints:**

- [x] ListKeys
- [x] CreateKey
- [x] ImportKey
- [x] GetKeyInfo
- [x] UpdateKey
- [x] DeleteKey

**Bucket-related endpoints:**

- [x] ListBuckets
- [x] CreateBucket
- [x] GetBucketInfo
- [x] DeleteBucket
- [x] PutBucketWebsite
- [x] DeleteBucketWebsite

**Operations on key/bucket permissions:**

- [x] BucketAllowKey
- [x] BucketDenyKey

**Operations on bucket aliases:**

- [x] GlobalAliasBucket
- [x] GlobalUnaliasBucket
- [x] LocalAliasBucket
- [x] LocalUnaliasBucket

**And also:**

- [x] Separate error type for the admin API (this PR includes a quite big refactoring of error handling)
- [x] Add management of website access
- [ ] Check that nothing is missing wrt what can be done using the CLI
- [ ] Improve formatting of the spec
- [x] Make sure everyone is cool with the API design

Fix #231
Fix #295

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: #298
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-05-24 12:16:39 +02:00
Alex 5768bf3622 First implementation of K2V (#293)
All checks were successful
continuous-integration/drone/push Build is passing
**Specification:**

View spec at [this URL](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/k2v/doc/drafts/k2v-spec.md)

- [x] Specify the structure of K2V triples
- [x] Specify the DVVS format used for causality detection
- [x] Specify the K2V index (just a counter of number of values per partition key)
- [x] Specify single-item endpoints: ReadItem, InsertItem, DeleteItem
- [x] Specify index endpoint: ReadIndex
- [x] Specify multi-item endpoints: InsertBatch, ReadBatch, DeleteBatch
- [x] Move to JSON objects instead of tuples
- [x] Specify endpoints for polling for updates on single values (PollItem)

**Implementation:**

- [x] Table for K2V items, causal contexts
- [x] Indexing mechanism and table for K2V index
- [x] Make API handlers a bit more generic
- [x] K2V API endpoint
- [x] K2V API router
- [x] ReadItem
- [x] InsertItem
- [x] DeleteItem
- [x] PollItem
- [x] ReadIndex
- [x] InsertBatch
- [x] ReadBatch
- [x] DeleteBatch

**Testing:**

- [x] Just a simple Python script that does some requests to check visually that things are going right (does not contain parsing of results or assertions on returned values)
- [x] Actual tests:
  - [x] Adapt testing framework
  - [x] Simple test with InsertItem + ReadItem
  - [x] Test with several Insert/Read/DeleteItem + ReadIndex
  - [x] Test all combinations of return formats for ReadItem
  - [x] Test with ReadBatch, InsertBatch, DeleteBatch
  - [x] Test with PollItem
  - [x] Test error codes
- [ ] Fix most broken stuff
  - [x] test PollItem broken randomly
  - [x] when invalid causality tokens are given, errors should be 4xx not 5xx

**Improvements:**

- [x] Descending range queries
  - [x] Specify
  - [x] Implement
  - [x] Add test
- [x] Batch updates to index counter
- [x] Put K2V behind `k2v` feature flag

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: #293
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-05-10 13:16:57 +02:00
Alex def78c5e6f
Update netapp to 0.4.4, fix #300
All checks were successful
continuous-integration/drone/pr Build is passing
continuous-integration/drone/push Build is passing
2022-05-09 12:08:47 +02:00