trinity-1686a/garage
2
0
Fork 0
forked from Deuxfleurs/garage
Code Issues Pull requests 1 Projects Releases Wiki Activity
1547 commits 13 branches 0 tags 59 MiB
Commit graph

52 commits

Author SHA1 Message Date
Alex 0c431b0c03 admin api: increased compatibility for v0/ endpoints 2023-10-05 16:56:13 +02:00
Alex 1c13135f25 admin api: remove broken GET /v0/key router rule 2023-10-05 16:27:29 +02:00
Alex 2e656b541b Merge branch 'main' into next 2023-10-03 18:40:37 +02:00
networkException 7907a09acc
api: allow custom unix bind mode and use 0o220 for admin server 2023-10-03 17:31:40 +02:00
networkException 8ec6a53b35
everywhere: support unix sockets when binding in various places 
this patch implements binding to paths as a unix socket for generic
server and web server.
 2023-09-29 18:57:44 +02:00
Alex 2e90e1c124 Merge branch 'main' into next 2023-08-29 11:32:42 +02:00
Alex 9b4ce4a8ad admin api: refactor caddy check api code 2023-08-28 12:17:10 +02:00
Quentin 24e533f262
support {s3,web}.root_domains in /check endpoint 2023-08-08 11:05:42 +02:00
Alex 8ef42c9609 admin docs: reformatting, key admin: add check 2023-06-14 17:19:25 +02:00
Alex a83a092c03 admin: uniformize layout api and improve code 2023-06-14 17:12:37 +02:00
Alex 7895f99d3a admin and cli: hide secret keys unless asked 2023-06-14 16:56:15 +02:00
Alex 4a82f6380e admin api: move all endpoints to v1/ by default (v0/ still supported) 2023-06-14 14:15:51 +02:00
Alex 28cc9f178a admin api: make name optionnal for CreateKey 2023-06-14 13:56:37 +02:00
Alex 35c108b85d admin api: switch GetClusterHealth to camelcase (fix #381 again) 2023-06-14 13:53:19 +02:00
Alex 52376d47ca admin api: change cluster status/layout to use lists and not maps (fix #377) 2023-06-14 13:45:27 +02:00
Alex 90b2d43eb4 Merge branch 'main' into next 2023-06-13 17:14:11 +02:00
Alex bf19a44fd9 admin API: add missing camelCase conversions (fix #381) 2023-06-13 16:15:50 +02:00
Alex 412ab77b08 comments and clippy lint fixes 2023-06-09 16:23:37 +02:00
Alex 511e07ecd4 fix mpu counter (add missing workers) and report info at appropriate places 2023-06-09 16:23:37 +02:00
Jonathan Davies c783194e8b *: apply clippy recommendations. 2023-05-09 20:49:34 +01:00
Alex fa78d806e3 Merge branch 'main' into next 2023-04-25 12:34:26 +02:00
Jonathan Davies 9ea154ae9c admin/cluster.rs: Added rust_version. 2023-03-10 14:46:54 +00:00
Jonathan Davies 9c354f0a8f Improved bucket authorization response strings. 2023-01-29 20:34:41 +00:00
Jonathan Davies 004bb5b4f1 api_server.rs: Adapted to use query string per Caddy upstream change. 2023-01-29 20:34:37 +00:00
Alex 8e93d69974 More clippy fixes 2023-01-26 17:26:32 +01:00
Jonathan Davies 100b01e859 Implemented website hosting authorization endpoint. 
Fixes: #468
 2023-01-13 09:38:58 +00:00
Alex 570e5e5bbb
Merge branch 'main' into next 2023-01-04 11:34:43 +01:00
Alex a0abf41762
Fix router keyword handling (fix #442) 2022-12-12 12:05:37 +01:00
Alex 2c2e65ad8b
Merge commit 'ec12d6c' into next 2022-12-11 18:41:15 +01:00
Alex 533afcf4e1
simplify 2022-12-11 18:17:08 +01:00
Alex 5ea5fd2130
Always return 200 OK on /v0/health, reinstate admin api doc as draft and complete it 2022-12-11 18:11:28 +01:00
Alex d7868c48a4
Separate /health (simple text answer) and /v0/health (full json answer, authenticated) 2022-12-05 15:38:32 +01:00
Alex 280d1be7b1
Refactor health check and add ability to return it in json 2022-12-05 15:28:57 +01:00
Alex 2065f011ca
Implement /health admin API endpoint to check node health 2022-12-05 14:59:15 +01:00
Quentin dc50fa3b34
Fix typo in admin API on BucketInfo 2022-11-11 16:56:56 +01:00
Alex 217abdca18
Fix HTTP return code 2022-11-08 15:38:53 +01:00
Alex d75b37b018
Return more info when layout's .check() fails, fix compilation, fix test 2022-11-08 14:58:39 +01:00
Alex ea5afc2511
Style improvements 2022-11-07 20:11:30 +01:00
Alex 28d7a49f63
Merge branch 'main' into optimal-layout 2022-11-07 12:20:59 +01:00
Tobias Krischer 7865003323
Use status code 204 No Content for empty responses 2022-10-17 10:55:26 +02:00
Mendes 4abab246f1 cargo fmt 2022-10-10 17:21:13 +02:00
Mendes ceac3713d6 modifications in several files to : 
- have consistent error return types
- store the zone redundancy in a Lww
- print the error and message in the CLI (TODO: for the server Api, should msg be returned in the body response?)
 2022-10-05 15:29:48 +02:00
Alex 782630fc27
Initialize metrics exporter earlier (fix #389) 2022-09-20 17:50:22 +02:00
Alex ceb1f0229a
Move version back into util 2022-09-07 18:36:46 +02:00
Alex 2559f63e9b
Make all HTTP services optionnal 2022-09-07 17:54:16 +02:00
Alex 28d86e7602
Report build features in garage --help 2022-09-07 17:05:21 +02:00
Alex db61f41030
Move GIT_VERSION injection later in build chain to reduce build times 2022-09-07 11:59:56 +02:00
Jakub Jirutka ea36b9ff90 Allow building without Prometheus exporter (/metrics endpoint) 
prometheus and opentelemetry-prometheus add 7 extra dependencies in
total and increases the size of the garage binary by ~7 % (with
fat LTO).
 2022-09-06 01:15:09 +02:00
Alex 77e3fd6db2 improve internal item counter mechanisms and implement bucket quotas (#326) 
- [x] Refactoring of internal counting API
- [x] Repair procedure for counters (it's an offline procedure!!!)
- [x] New counter for objects in buckets
- [x] Add quotas to buckets struct
- [x] Add CLI to manage bucket quotas
- [x] Add admin API to manage bucket quotas
- [x] Apply quotas by adding checks on put operations
- [x] Proof-read

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: Deuxfleurs/garage#326
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
 2022-06-15 20:20:28 +02:00
Alex b44d3fc796 Abstract database behind generic interface and implement alternative drivers (#322) 
- [x] Design interface
- [x] Implement Sled backend
  - [x] Re-implement the SledCountedTree hack ~~on Sled backend~~ on all backends (i.e. over the abstraction)
- [x] Convert Garage code to use generic interface
- [x] Proof-read converted Garage code
- [ ] Test everything well
- [x] Implement sqlite backend
- [x] Implement LMDB backend
- [ ] (Implement Persy backend?)
- [ ] (Implement other backends? (like RocksDB, ...))
- [x] Implement backend choice in config file and garage server module
- [x] Add CLI for converting between DB formats
- Exploit the new interface to put more things in transactions
  - [x] `.updated()` trigger on Garage tables

Fix #284

**Bugs**

- [x] When exporting sqlite, trees iterate empty??
- [x] LMDB doesn't work

**Known issues for various back-ends**

- Sled:
  - Eats all my RAM and also all my disk space
  - `.len()` has to traverse the whole table
  - Is actually quite slow on some operations
  - And is actually pretty bad code...
- Sqlite:
  - Requires a lock to be taken on all operations. The lock is also taken when iterating on a table with `.iter()`, and the lock isn't released until the iterator is dropped. This means that we must be VERY carefull to not do anything else inside a `.iter()` loop or else we will have a deadlock! Most such cases have been eliminated from the Garage codebase, but there might still be some that remain. If your Garage-over-Sqlite seems to hang/freeze, this is the reason.
  - (adapter uses a bunch of unsafe code)
- Heed (LMDB):
  - Not suited for 32-bit machines as it has to map the whole DB in memory.
  - (adpater uses a tiny bit of unsafe code)

**My recommendation:** avoid 32-bit machines and use LMDB as much as possible.

**Converting databases** is actually quite easy. For example from Sled to LMDB:

```bash
cd src/db
cargo run --features cli --bin convert -- -i path/to/garage/meta/db -a sled -o path/to/garage/meta/db.lmdb -b lmdb
```

Then, just add this to your `config.toml`:

```toml
db_engine = "lmdb"
```

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: Deuxfleurs/garage#322
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
 2022-06-08 10:01:44 +02:00