Commit graph

22 commits

Author SHA1 Message Date
Jonathan Davies
9c354f0a8f Improved bucket authorization response strings. 2023-01-29 20:34:41 +00:00
Jonathan Davies
004bb5b4f1 api_server.rs: Adapted to use query string per Caddy upstream change. 2023-01-29 20:34:37 +00:00
8e93d69974 More clippy fixes 2023-01-26 17:26:32 +01:00
Jonathan Davies
100b01e859 Implemented website hosting authorization endpoint.
Fixes: 
2023-01-13 09:38:58 +00:00
a0abf41762
Fix router keyword handling (fix ) 2022-12-12 12:05:37 +01:00
533afcf4e1
simplify 2022-12-11 18:17:08 +01:00
5ea5fd2130
Always return 200 OK on /v0/health, reinstate admin api doc as draft and complete it 2022-12-11 18:11:28 +01:00
d7868c48a4
Separate /health (simple text answer) and /v0/health (full json answer, authenticated) 2022-12-05 15:38:32 +01:00
280d1be7b1
Refactor health check and add ability to return it in json 2022-12-05 15:28:57 +01:00
2065f011ca
Implement /health admin API endpoint to check node health 2022-12-05 14:59:15 +01:00
dc50fa3b34
Fix typo in admin API on BucketInfo 2022-11-11 16:56:56 +01:00
7865003323
Use status code 204 No Content for empty responses 2022-10-17 10:55:26 +02:00
782630fc27
Initialize metrics exporter earlier (fix ) 2022-09-20 17:50:22 +02:00
ceb1f0229a
Move version back into util 2022-09-07 18:36:46 +02:00
2559f63e9b
Make all HTTP services optionnal 2022-09-07 17:54:16 +02:00
28d86e7602
Report build features in garage --help 2022-09-07 17:05:21 +02:00
db61f41030
Move GIT_VERSION injection later in build chain to reduce build times 2022-09-07 11:59:56 +02:00
ea36b9ff90 Allow building without Prometheus exporter (/metrics endpoint)
prometheus and opentelemetry-prometheus add 7 extra dependencies in
total and increases the size of the garage binary by ~7 % (with
fat LTO).
2022-09-06 01:15:09 +02:00
77e3fd6db2 improve internal item counter mechanisms and implement bucket quotas ()
- [x] Refactoring of internal counting API
- [x] Repair procedure for counters (it's an offline procedure!!!)
- [x] New counter for objects in buckets
- [x] Add quotas to buckets struct
- [x] Add CLI to manage bucket quotas
- [x] Add admin API to manage bucket quotas
- [x] Apply quotas by adding checks on put operations
- [x] Proof-read

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: 
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-06-15 20:20:28 +02:00
b44d3fc796 Abstract database behind generic interface and implement alternative drivers ()
- [x] Design interface
- [x] Implement Sled backend
  - [x] Re-implement the SledCountedTree hack ~~on Sled backend~~ on all backends (i.e. over the abstraction)
- [x] Convert Garage code to use generic interface
- [x] Proof-read converted Garage code
- [ ] Test everything well
- [x] Implement sqlite backend
- [x] Implement LMDB backend
- [ ] (Implement Persy backend?)
- [ ] (Implement other backends? (like RocksDB, ...))
- [x] Implement backend choice in config file and garage server module
- [x] Add CLI for converting between DB formats
- Exploit the new interface to put more things in transactions
  - [x] `.updated()` trigger on Garage tables

Fix 

**Bugs**

- [x] When exporting sqlite, trees iterate empty??
- [x] LMDB doesn't work

**Known issues for various back-ends**

- Sled:
  - Eats all my RAM and also all my disk space
  - `.len()` has to traverse the whole table
  - Is actually quite slow on some operations
  - And is actually pretty bad code...
- Sqlite:
  - Requires a lock to be taken on all operations. The lock is also taken when iterating on a table with `.iter()`, and the lock isn't released until the iterator is dropped. This means that we must be VERY carefull to not do anything else inside a `.iter()` loop or else we will have a deadlock! Most such cases have been eliminated from the Garage codebase, but there might still be some that remain. If your Garage-over-Sqlite seems to hang/freeze, this is the reason.
  - (adapter uses a bunch of unsafe code)
- Heed (LMDB):
  - Not suited for 32-bit machines as it has to map the whole DB in memory.
  - (adpater uses a tiny bit of unsafe code)

**My recommendation:** avoid 32-bit machines and use LMDB as much as possible.

**Converting databases** is actually quite easy. For example from Sled to LMDB:

```bash
cd src/db
cargo run --features cli --bin convert -- -i path/to/garage/meta/db -a sled -o path/to/garage/meta/db.lmdb -b lmdb
```

Then, just add this to your `config.toml`:

```toml
db_engine = "lmdb"
```

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: 
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-06-08 10:01:44 +02:00
ff06d3f082
Fix Content-Type headers for {admin,k2v} errors and admin responses
Fix 
2022-05-25 17:09:33 +02:00
382e74c798 First version of admin API ()
**Spec:**

- [x] Start writing
- [x] Specify all layout endpoints
- [x] Specify all endpoints for operations on keys
- [x] Specify all endpoints for operations on key/bucket permissions
- [x] Specify all endpoints for operations on buckets
- [x] Specify all endpoints for operations on bucket aliases

View rendered spec at <https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/admin-api/doc/drafts/admin-api.md>

**Code:**

- [x] Refactor code for admin api to use common api code that was created for K2V

**General endpoints:**

- [x] Metrics
- [x] GetClusterStatus
- [x] ConnectClusterNodes
- [x] GetClusterLayout
- [x] UpdateClusterLayout
- [x] ApplyClusterLayout
- [x] RevertClusterLayout

**Key-related endpoints:**

- [x] ListKeys
- [x] CreateKey
- [x] ImportKey
- [x] GetKeyInfo
- [x] UpdateKey
- [x] DeleteKey

**Bucket-related endpoints:**

- [x] ListBuckets
- [x] CreateBucket
- [x] GetBucketInfo
- [x] DeleteBucket
- [x] PutBucketWebsite
- [x] DeleteBucketWebsite

**Operations on key/bucket permissions:**

- [x] BucketAllowKey
- [x] BucketDenyKey

**Operations on bucket aliases:**

- [x] GlobalAliasBucket
- [x] GlobalUnaliasBucket
- [x] LocalAliasBucket
- [x] LocalUnaliasBucket

**And also:**

- [x] Separate error type for the admin API (this PR includes a quite big refactoring of error handling)
- [x] Add management of website access
- [ ] Check that nothing is missing wrt what can be done using the CLI
- [ ] Improve formatting of the spec
- [x] Make sure everyone is cool with the API design

Fix 
Fix 

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: 
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-05-24 12:16:39 +02:00