Commit Graph

41 Commits

Author SHA1 Message Date
Felix Scheinost f2106c2733 Implement `rpc_secret_file`
continuous-integration/drone/pr Build is passing Details
2023-01-04 18:35:10 +01:00
Alex de9d6cddf7
Prettier worker list table; remove useless CLI log messages
continuous-integration/drone/push Build is passing Details
2022-12-12 17:17:05 +01:00
Alex e03d9062f7
Show a nice message and a backtrace when Garage panics
continuous-integration/drone/pr Build is passing Details
continuous-integration/drone/push Build is passing Details
2022-11-04 16:39:02 +01:00
Alex 002b9fc50c
Add TLS support for Consul discovery + refactoring
continuous-integration/drone/pr Build is passing Details
continuous-integration/drone/push Build is passing Details
2022-10-18 18:38:20 +02:00
Alex e46dc2a8ef
Allow for hostnames in bootstrap_peers and rpc_public_addr (fix #353)
continuous-integration/drone/pr Build is passing Details
continuous-integration/drone/push Build is passing Details
2022-09-14 16:09:38 +02:00
Alex d9d199a6c9
Merge branch 'main' into lx-perf-improvements
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/pr Build is passing Details
2022-09-08 15:49:17 +02:00
Alex ceb1f0229a
Move version back into util
continuous-integration/drone/pr Build is passing Details
continuous-integration/drone/push Build is passing Details
2022-09-07 18:36:46 +02:00
Alex f310fce34b
Inject GIT_VERSION even later
continuous-integration/drone/pr Build is passing Details
continuous-integration/drone/push Build is passing Details
2022-09-07 18:30:15 +02:00
Alex 2e00809af5
Error messages when system-libs XOR bundled-libs != 1
continuous-integration/drone/push Build is pending Details
continuous-integration/drone/pr Build is pending Details
2022-09-07 17:57:12 +02:00
Alex 28d86e7602
Report build features in garage --help
continuous-integration/drone/pr Build is failing Details
continuous-integration/drone/push Build is passing Details
2022-09-07 17:05:21 +02:00
Alex db61f41030
Move GIT_VERSION injection later in build chain to reduce build times
continuous-integration/drone/pr Build is failing Details
continuous-integration/drone/push Build is passing Details
2022-09-07 11:59:56 +02:00
Jakub Jirutka e7af006c1c Make OTLP exporter optional via feature "telemetry-otlp"
opentelemetry-otlp add 48 (!) extra dependencies and increases the
size of the garage binary by ~11 % (with fat LTO).
2022-09-06 01:14:47 +02:00
Alex efbca67ce4
Add env filter to tracing subscriber
continuous-integration/drone/pr Build is failing Details
continuous-integration/drone/push Build is failing Details
2022-08-31 14:39:12 +02:00
Alex 44cd98d2e4
Tracing-subscriber: write to stderr
continuous-integration/drone/pr Build is passing Details
continuous-integration/drone/push Build is passing Details
2022-08-31 14:28:17 +02:00
Alex dd5304f6fc
Replace logging crate pretty_env_logger by tracing_subscriber::fmt
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/pr Build is passing Details
2022-08-31 14:24:41 +02:00
Quentin 2c7bae935a
Configure structopt to report the right version
continuous-integration/drone/pr Build is passing Details
continuous-integration/drone/tag Build is passing Details
continuous-integration/drone Build is passing Details
continuous-integration/drone/push Build is passing Details
By default, structopt reports the value provided by
the env var CARGO_PKG_VERSION, feeded by Cargo when reading
Cargo.toml. However for Garage we use a versioning based on git,
so we often report a version that is behind the real version.
In this commit, we create garage_util::version::garage() that
reports the right version and configure all structopt subcommands
to call this function instead of using the env var.
2022-08-11 10:21:45 +02:00
Alex 77e3fd6db2 improve internal item counter mechanisms and implement bucket quotas (#326)
continuous-integration/drone/push Build is passing Details
- [x] Refactoring of internal counting API
- [x] Repair procedure for counters (it's an offline procedure!!!)
- [x] New counter for objects in buckets
- [x] Add quotas to buckets struct
- [x] Add CLI to manage bucket quotas
- [x] Add admin API to manage bucket quotas
- [x] Apply quotas by adding checks on put operations
- [x] Proof-read

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: #326
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-06-15 20:20:28 +02:00
Alex 382e74c798 First version of admin API (#298)
continuous-integration/drone/push Build is passing Details
**Spec:**

- [x] Start writing
- [x] Specify all layout endpoints
- [x] Specify all endpoints for operations on keys
- [x] Specify all endpoints for operations on key/bucket permissions
- [x] Specify all endpoints for operations on buckets
- [x] Specify all endpoints for operations on bucket aliases

View rendered spec at <https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/admin-api/doc/drafts/admin-api.md>

**Code:**

- [x] Refactor code for admin api to use common api code that was created for K2V

**General endpoints:**

- [x] Metrics
- [x] GetClusterStatus
- [x] ConnectClusterNodes
- [x] GetClusterLayout
- [x] UpdateClusterLayout
- [x] ApplyClusterLayout
- [x] RevertClusterLayout

**Key-related endpoints:**

- [x] ListKeys
- [x] CreateKey
- [x] ImportKey
- [x] GetKeyInfo
- [x] UpdateKey
- [x] DeleteKey

**Bucket-related endpoints:**

- [x] ListBuckets
- [x] CreateBucket
- [x] GetBucketInfo
- [x] DeleteBucket
- [x] PutBucketWebsite
- [x] DeleteBucketWebsite

**Operations on key/bucket permissions:**

- [x] BucketAllowKey
- [x] BucketDenyKey

**Operations on bucket aliases:**

- [x] GlobalAliasBucket
- [x] GlobalUnaliasBucket
- [x] LocalAliasBucket
- [x] LocalUnaliasBucket

**And also:**

- [x] Separate error type for the admin API (this PR includes a quite big refactoring of error handling)
- [x] Add management of website access
- [ ] Check that nothing is missing wrt what can be done using the CLI
- [ ] Improve formatting of the spec
- [x] Make sure everyone is cool with the API design

Fix #231
Fix #295

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: #298
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-05-24 12:16:39 +02:00
Alex ba6b56ae68
Fix some new clippy lints
continuous-integration/drone/pr Build is passing Details
continuous-integration/drone/push Build is passing Details
2022-03-14 12:27:49 +01:00
Alex bb04d94fa9
Update to Netapp 0.4 which supports distributed tracing 2022-03-14 10:52:30 +01:00
Alex 8c2fb0c066
Add tracing integration with opentelemetry 2022-03-14 10:52:13 +01:00
Alex df35feba18
New buckets for 0.6.0: make bucket id a SK and not a HK, CLI updates
continuous-integration/drone/pr Build is passing Details
continuous-integration/drone/push Build is passing Details
2022-01-04 12:53:14 +01:00
Alex beeef4758e
Some movement of helper code and refactoring of error handling 2022-01-04 12:52:46 +01:00
Alex c94406f428
Improve how node roles are assigned in Garage
continuous-integration/drone/pr Build is passing Details
continuous-integration/drone/tag Build is passing Details
continuous-integration/drone/push Build is passing Details
continuous-integration/drone Build is passing Details
- change the terminology: the network configuration becomes the role
  table, the configuration of a nodes becomes a node's role
- the modification of the role table takes place in two steps: first,
  changes are staged in a CRDT data structure. Then, once the user is
  happy with the changes, they can commit them all at once (or revert
  them).
- update documentation
- fix tests
- implement smarter partition assignation algorithm

This patch breaks the format of the network configuration: when
migrating, the cluster will be in a state where no roles are assigned.
All roles must be re-assigned and commited at once. This migration
should not pose an issue.
2021-11-16 16:05:53 +01:00
Jill bef6d627b0 Add environment variables equivalents for some CLI options.
continuous-integration/drone/push Build is passing Details
2021-11-03 16:00:57 +01:00
Alex 3e7f766d95
CLI: default rpc_host
continuous-integration/drone/pr Build is passing Details
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/tag Build is passing Details
continuous-integration/drone Build is passing Details
2021-10-26 11:36:30 +02:00
Alex de4276202a
Improve CLI, adapt tests, update documentation 2021-10-25 14:21:48 +02:00
Alex 1b450c4b49
Improvements to CLI and various fixes for netapp version
Discovery via consul, persist peer list to file
2021-10-22 16:55:24 +02:00
Alex 4067797d01
First port of Garage to Netapp 2021-10-22 15:55:18 +02:00
Trinity Pointard b568765c75 try parsing rpc-host command-line parameter
continuous-integration/drone/push Build is passing Details
2021-06-01 23:53:58 +02:00
Trinity Pointard 74373aebcf
make most requested changes 2021-04-27 16:47:08 +02:00
Trinity Pointard 5c4b2cec3c
document garage crate 2021-04-27 16:37:10 +02:00
Alex f4aad8fe6e cargo fmt
continuous-integration/drone/push Build is passing Details
2021-03-12 18:16:03 +01:00
Alex 5ab33fddac Refactor CLI and prettify CLI outpu
continuous-integration/drone/push Build is passing Details
2021-03-12 18:12:31 +01:00
Alex a1442f072a Implement garage stats to get info on node contents
continuous-integration/drone/push Build is passing Details
2021-03-12 15:40:54 +01:00
Alex 6a3dcf3974 Rename n_tokens into capacity
continuous-integration/drone/push Build is passing Details
2021-03-10 14:52:03 +01:00
Alex 80892df8cc Some refactoring
continuous-integration/drone/push Build is passing Details
2021-02-21 13:11:10 +01:00
Quentin a3566e49da Start to implement Website CLI 2020-12-14 21:50:40 +01:00
Quentin e1ce2b228a WIP table migration 2020-12-12 17:00:31 +01:00
Quentin 51d0c14e44 CLI structure 2020-12-10 18:13:32 +01:00
Alex d8f5e643bc Split code for modular compilation 2020-04-24 10:10:01 +00:00
Renamed from src/main.rs (Browse further)