admin API refactoring (step 1) #939

Merged

lx merged 19 commits from refactor-admin into next-v2 2025-01-29 20:42:57 +00:00

Conversation 0 Commits 19 Files changed 22 +3238 -1147

lx commented 2025-01-27 23:25:34 +00:00

Owner

Copy link

The new API syntax is somewhat inspired by XRPC: we are no longer trying to be RESTful or to guess API endpoints by HTTP verb or query parameters, the name of the API endpoint is directly in the path (ex: /v2/ListBuckets).

The aim is to add to the HTTP Admin API all necessary verbs to be able to rewrite the CLI code using only these verbs. In a first step, the CLI will proxy these calls within the RPC protocol so as not to disrupt the current way it works. The next step is to make it able to do these calls directly to the HTTP Admin endpoint, using the (an) admin token. Combined with multiple Admin API tokens (also planned for Garage v2.0) this will make the CLI more flexible and powerful.

• Unify structs for all API calls
• Make handlers impls of a trait on the *Request structs
• Change request syntax to be uniform
• Make into macros
• Convert v1/ endpoints to v2/ when possible
• Update API documentation in drafts/admin-api.md
• Update OpenAPI specification
• (Rebuild client SDKs) (after release ?)
• Proof-reading
• Testing

For step 2:

• Make admin API calls available through internal RPC and use them in the CLI as much as possible
• Convert any functionnality that was only in the CLI into now-standardized admin API calls
• Update OpenAPI specification again
• Make the CLI able to call the admin API through HTTP as well

---

• GetKeyInfo 404 if not found
• search for getkeyinfo but not getbucketinfo? -> deprecate it for getkeyinfo, and move to new special endpoints SearchKey and SearchBucket that can return multiple results
• fix test_website_check_domain

The new API syntax is somewhat inspired by [XRPC](https://atproto.com/specs/xrpc): we are no longer trying to be RESTful or to guess API endpoints by HTTP verb or query parameters, the name of the API endpoint is directly in the path (ex: `/v2/ListBuckets`). The aim is to add to the HTTP Admin API all necessary verbs to be able to rewrite the CLI code using only these verbs. In a first step, the CLI will proxy these calls within the RPC protocol so as not to disrupt the current way it works. The next step is to make it able to do these calls directly to the HTTP Admin endpoint, using the (an) admin token. Combined with multiple Admin API tokens (also planned for Garage v2.0) this will make the CLI more flexible and powerful. - [x] Unify structs for all API calls - [x] Make handlers impls of a trait on the *Request structs - [x] Change request syntax to be uniform - [x] Make into macros - [x] Convert v1/ endpoints to v2/ when possible - [x] Update API documentation in `drafts/admin-api.md` - [x] Update OpenAPI specification - [ ] (Rebuild client SDKs) (after release ?) - [x] Proof-reading - [x] Testing For step 2: - [ ] Make admin API calls available through internal RPC and use them in the CLI as much as possible - [ ] Convert any functionnality that was only in the CLI into now-standardized admin API calls - [ ] Update OpenAPI specification again - [ ] Make the CLI able to call the admin API through HTTP as well --- - [x] GetKeyInfo 404 if not found - [ ] search for getkeyinfo but not getbucketinfo? -> deprecate it for getkeyinfo, and move to new special endpoints SearchKey and SearchBucket that can return multiple results - [x] fix test_website_check_domain

lx added 2 commits 2025-01-27 23:25:34 +00:00

admin api: create structs for all requests/responess in src/api/admin/api.rs 712ab0c768

admin api: make all handlers impls of a single trait f138630294

lx added this to the v2.0 milestone 2025-01-27 23:25:39 +00:00

lx force-pushed refactor-admin from f138630294 to 4533b08f85 2025-01-27 23:28:05 +00:00 Compare

lx added 1 commit 2025-01-28 14:12:21 +00:00

admin api: new router_v2 with unified path syntax a67c1262ca

lx force-pushed refactor-admin from a67c1262ca to f81c2333f1 2025-01-28 14:16:40 +00:00 Compare

lx added 1 commit 2025-01-28 14:44:24 +00:00

admin api: refactor using macro 5eea6d3f0a

lx added 2 commits 2025-01-28 15:19:02 +00:00

admin api: add compatibility from v1/ to v2/ 9beb66a63c

admin api: update semantics of some endpoints, and update doc 2abfd4fb3a

lx added 1 commit 2025-01-28 16:07:45 +00:00

admin api: make all requests and responses (de)serializable ae755af563

lx force-pushed refactor-admin from ae755af563 to eeb08e2225 2025-01-28 16:10:45 +00:00 Compare

lx force-pushed refactor-admin from eeb08e2225 to 2c95d4816f 2025-01-28 16:24:16 +00:00 Compare

lx added 2 commits 2025-01-28 16:56:38 +00:00

admin api: initialize v2 openapi spec from v1 3b0c190b08

admin api: wip v2 openapi spec e4ab7c60f2

lx force-pushed refactor-admin from e4ab7c60f2 to b2c1d6b045 2025-01-28 17:02:43 +00:00 Compare

lx force-pushed refactor-admin from b2c1d6b045 to 7719f6a5b1 2025-01-28 17:03:20 +00:00 Compare

lx added 1 commit 2025-01-28 17:15:46 +00:00

admin api: fix CORS to work in browser 50761a51b9

lx force-pushed refactor-admin from 50761a51b9 to d15bea9380 2025-01-28 17:29:05 +00:00 Compare

lx added 1 commit 2025-01-28 17:40:48 +00:00

fix test_website_check_domain 17cbfb7fa6

lx added 2 commits 2025-01-28 17:53:58 +00:00

admin api: rename bucket aliasing operations 70bf358b60

admin api: rename allow/deny api calls in api v2 348a9a4cd6

lx added 1 commit 2025-01-28 18:03:51 +00:00

admin api: merge calls to manage global/local aliases 35c69964ef

lx changed title from WIP: admin API refactoring to WIP: admin API refactoring (step 1) 2025-01-28 18:08:31 +00:00

lx added 1 commit 2025-01-29 10:06:59 +00:00

admin api: clearer syntax for AddBucketAlias and RemoveBucketAlias 79a81f81cf

lx force-pushed refactor-admin from 79a81f81cf to 6d4c1ed23a 2025-01-29 10:10:20 +00:00 Compare

lx added 1 commit 2025-01-29 11:07:03 +00:00

admin api: small fixes 8515060a08

lx force-pushed refactor-admin from 8515060a08 to 1c03941b19 2025-01-29 18:26:22 +00:00 Compare

lx added 1 commit 2025-01-29 18:47:49 +00:00

admin api: remove log message 19454c1679

lx changed title from WIP: admin API refactoring (step 1) to admin API refactoring (step 1) 2025-01-29 20:42:51 +00:00

lx merged commit 6ed78abb5c into next-v2 2025-01-29 20:42:57 +00:00

lx referenced this pull request from a commit 2025-01-29 20:42:59 +00:00

Merge pull request 'admin API refactoring (step 1)' (#939) from refactor-admin into next-v2

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

action

check-aws

The behaviour should be checked against AWS S3

  

action

discussion-needed

We need to discuss if the feature is desirable and/or better specify the envisioned behavior or technical implementation

  

action

for-external-contributors

Core team does not plan working on this in the near future however an external contribution will be reviewed & merged

  

action

for-newcomers

An issue that should be newcomer friendly.

  

action

more-info-needed

More information is needed to investigate this issue

  

action

need-funding

This issue represents lots of work, we need to find money to pay our core team work

  

action

triage-required

It's not very clear what we want to do with this issue yet, more thoughts is required

  

kind

correctness

Issues of theoretical correctness that may impact edge-case scenarios

  

kind

ideas

For abstract ideas/proposal to make Garage better, to better communicate our not yet achieved aims

  

kind

improvement

Feature requests that do not fall in other categories

  

kind

performance

Deceptive performance observed OR idea to improve performances

  

kind

testing

Functional tests, integration tests, Jepsen, etc.

  

kind

usability

Propositions to make Garage easier to operate, by improving error reporting, CLI, etc.

  

kind

wrong-behavior

Unintended behavior

  

prio

critical

Could impact integrity or availability of production clusters

  

prio

low

For issues that we don't want to spend time on until made absolutely necessary

  

scope

admin-api

Issues concerning the administration API

  

scope

background-healing

Background healing covers repair tasks, anti-entropy, merkle trees, etc. ; everything related to replicating data appropriately.

  

scope

build

Issues with Nix, Drone, Rust Compiler, etc.

  

scope

documentation

  

scope

k8s

Kubernetes related, including Helm Charts

  

scope

layout

Linked to the cluster layout and storage management

  

scope

metadata

Metadata Engine (LMDB, sqlite, etc.)

  

scope

ops

Operations related to operations/maintenance issue, tooling related (like CLI missing info ; impossible, too dangerous, too complicated migration, etc.)

  

scope

rpc

Linked to RPC

  

scope

s3-api

Related to the S3 API (especially compatibility issues).

  

scope

security

Cryptography-related, auth-related, authorization-related issues

  

scope

telemetry

OpenTelemetry (traces, metrics, etc.)

No labels

action

check-aws

action

discussion-needed

action

for-external-contributors

action

for-newcomers

action

more-info-needed

action

need-funding

action

triage-required

kind

correctness

kind

ideas

kind

improvement

kind

performance

kind

testing

kind

usability

kind

wrong-behavior

prio

critical

prio

low

scope

admin-api

scope

background-healing

scope

build

scope

documentation

scope

k8s

scope

layout

scope

metadata

scope

ops

scope

rpc

scope

s3-api

scope

security

scope

telemetry

Milestone

Clear milestone

No items

No milestone

v2.0

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Deuxfleurs/garage#939

No description provided.