more complete admin API #298

Merged

lx merged 48 commits from admin-api into main 2022-05-24 10:16:40 +00:00

Conversation 0 Commits 48 Files changed 65 +3253 -986

lx commented 2022-05-05 08:31:10 +00:00

Owner

Copy link

Spec:

• Start writing
• Specify all layout endpoints
• Specify all endpoints for operations on keys
• Specify all endpoints for operations on key/bucket permissions
• Specify all endpoints for operations on buckets
• Specify all endpoints for operations on bucket aliases

View rendered spec at https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/admin-api/doc/drafts/admin-api.md

Code:

• Refactor code for admin api to use common api code that was created for K2V

General endpoints:

• Metrics
• GetClusterStatus
• ConnectClusterNodes
• GetClusterLayout
• UpdateClusterLayout
• ApplyClusterLayout
• RevertClusterLayout

Key-related endpoints:

• ListKeys
• CreateKey
• ImportKey
• GetKeyInfo
• UpdateKey
• DeleteKey

Bucket-related endpoints:

• ListBuckets
• CreateBucket
• GetBucketInfo
• DeleteBucket
• PutBucketWebsite
• DeleteBucketWebsite

Operations on key/bucket permissions:

• BucketAllowKey
• BucketDenyKey

Operations on bucket aliases:

• GlobalAliasBucket
• GlobalUnaliasBucket
• LocalAliasBucket
• LocalUnaliasBucket

And also:

• Separate error type for the admin API (this PR includes a quite big refactoring of error handling)
• Add management of website access
• Check that nothing is missing wrt what can be done using the CLI
• Improve formatting of the spec
• Make sure everyone is cool with the API design

Fix #231
Fix #295

**Spec:** - [x] Start writing - [x] Specify all layout endpoints - [x] Specify all endpoints for operations on keys - [x] Specify all endpoints for operations on key/bucket permissions - [x] Specify all endpoints for operations on buckets - [x] Specify all endpoints for operations on bucket aliases View rendered spec at <https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/admin-api/doc/drafts/admin-api.md> **Code:** - [x] Refactor code for admin api to use common api code that was created for K2V **General endpoints:** - [x] Metrics - [x] GetClusterStatus - [x] ConnectClusterNodes - [x] GetClusterLayout - [x] UpdateClusterLayout - [x] ApplyClusterLayout - [x] RevertClusterLayout **Key-related endpoints:** - [x] ListKeys - [x] CreateKey - [x] ImportKey - [x] GetKeyInfo - [x] UpdateKey - [x] DeleteKey **Bucket-related endpoints:** - [x] ListBuckets - [x] CreateBucket - [x] GetBucketInfo - [x] DeleteBucket - [x] PutBucketWebsite - [x] DeleteBucketWebsite **Operations on key/bucket permissions:** - [x] BucketAllowKey - [x] BucketDenyKey **Operations on bucket aliases:** - [x] GlobalAliasBucket - [x] GlobalUnaliasBucket - [x] LocalAliasBucket - [x] LocalUnaliasBucket **And also:** - [x] Separate error type for the admin API (this PR includes a quite big refactoring of error handling) - [x] Add management of website access - [ ] Check that nothing is missing wrt what can be done using the CLI - [ ] Improve formatting of the spec - [x] Make sure everyone is cool with the API design Fix #231 Fix #295

lx force-pushed admin-api from 60af164433 to d19ed12f92 2022-05-09 09:13:46 +00:00 Compare

lx referenced this pull request 2022-05-09 09:15:20 +00:00

Don't change output of garage layout show if staged layout is the same as the current #295

lx changed target branch from k2v to main 2022-05-10 11:13:36 +00:00

lx added 1 commit 2022-05-10 11:15:51 +00:00

Merge branch 'main' into k2v aa5b54fe19

lx force-pushed admin-api from e97d24e008 to dd54d0b2b1 2022-05-10 11:25:20 +00:00 Compare

lx added 1 commit 2022-05-10 11:36:49 +00:00

Update Cargo.nix and improve log message bb6ec9ebd9

lx added 1 commit 2022-05-11 08:28:06 +00:00

Add API access key admin endpoints f97a7845e9

lx added 1 commit 2022-05-11 09:10:41 +00:00

First key endpoints: ListKeys and GetKeyInfo 5c00c9fb46

lx added 1 commit 2022-05-11 09:40:44 +00:00

Implement CreateKey, DeleteKey and rudimentary UpdateKey 393b76ecba

lx added 1 commit 2022-05-11 09:51:23 +00:00

Short doc on UpdateKey aeb978552a

lx added 1 commit 2022-05-12 08:22:52 +00:00

ListBucket and GetBucketInfo 2b93a01d2b

lx added 1 commit 2022-05-12 08:45:21 +00:00

CreateBucket de1a5b87b6

lx added 1 commit 2022-05-12 09:02:48 +00:00

DeleteBucket fe399a3265

lx added 1 commit 2022-05-12 09:19:53 +00:00

BucketAllowKey and BucketDenyKey fc2f73ddb5

lx added 1 commit 2022-05-12 09:21:31 +00:00

Simplify ed76893581

lx added 2 commits 2022-05-12 15:12:27 +00:00

Slightly more detailed error reporting from helper e7ddba53e3

Fix clippy e4e1f8f0d6

lx added 3 commits 2022-05-13 13:05:16 +00:00

Possibility of different error types for different APIs 983037d965

Common error type and admin error type that uses it c0fb9fd0fe

Error refactoring 96b11524d5

lx added 2 commits 2022-05-13 13:21:50 +00:00

Rename error::Error to s3::error::Error f82b938033

More error refactoring 7a5d329e49

lx added 1 commit 2022-05-13 13:44:52 +00:00

Separate error types for k2v and signature ec16d166f9

lx added 1 commit 2022-05-13 17:19:06 +00:00

More error refactoring ea325d78d3

lx added 1 commit 2022-05-13 17:28:38 +00:00

Json body for custom errors 5a535788fc

lx added 1 commit 2022-05-13 17:36:31 +00:00

More precisions in errors & small refactoring 8033bdb0b4

lx added 1 commit 2022-05-13 17:43:56 +00:00

Revert useless thing d7736cb614

lx added 1 commit 2022-05-13 17:49:16 +00:00

Remove useless string conversions ec50ffac42

lx added 4 commits 2022-05-17 11:11:03 +00:00

Fix ReadIndex spec and add JSON5 remark to doc 176715c5b2

Make background runner terminate correctly 7b474855e3

K2V: Fix end parameter and add tests (fix #305) c692f55d5c

Merge branch 'main' into admin-api 86a3fe8ec5

lx added 1 commit 2022-05-17 14:43:10 +00:00

Return website config in GetBucketInfo, use serde(rename_all) 8ff95f09c9

lx added 1 commit 2022-05-17 14:48:42 +00:00

Fix error code 8b1338ef2f

lx added 2 commits 2022-05-17 15:17:01 +00:00

refactor e92c52eb65

Specify and implement {Global,Local}{Alias,Unalias}Bucket 2ce3513c10

lx added 2 commits 2022-05-17 15:52:18 +00:00

Add PutBucketWebsite and DeleteBucketWebsite to admin api 5072dbd228

Prefix all APIs with v0/ 440a577563

lx added 1 commit 2022-05-17 16:44:06 +00:00

Implement ConnectClusterNodes 70383b4363

lx added 1 commit 2022-05-17 17:02:38 +00:00

Implement ImportKey dcfa408887

lx added 1 commit 2022-05-17 22:28:31 +00:00

Rename error varian for Clippy 926b3c0fad

lx added 1 commit 2022-05-17 22:33:06 +00:00

Fix fmt 30e393b439

lx added 1 commit 2022-05-18 08:10:34 +00:00

Refactor bucket emptiness check and add k2v check 5367f8adb2

lx added 1 commit 2022-05-18 20:36:04 +00:00

Merge branch 'main' into admin-api 99976e11f8

lx added 1 commit 2022-05-23 10:06:23 +00:00

Update documentation with warning d768f559da

lx added 1 commit 2022-05-23 14:40:37 +00:00

Make authorization token mandatory for admin API 1c88ee9bc5

lx added 1 commit 2022-05-23 15:39:33 +00:00

fix doc 2f250a83e1

lx added 1 commit 2022-05-24 09:53:09 +00:00

Fix some docs and change syntax of CreateBucket permissions 0b43a71351

lx added 1 commit 2022-05-24 09:54:04 +00:00

cargo fmt be59cafd47

lx changed title from WIP: more complete admin API to more complete admin API 2022-05-24 10:16:01 +00:00

lx merged commit 382e74c798 into main 2022-05-24 10:16:40 +00:00

lx referenced this pull request from a commit 2022-05-24 10:16:41 +00:00

First version of admin API (#298)

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

action

check-aws

The behaviour should be checked against AWS S3

  

action

discussion-needed

We need to discuss if the feature is desirable and/or better specify the envisioned behavior or technical implementation

  

action

for-external-contributors

Core team does not plan working on this in the near future however an external contribution will be reviewed & merged

  

action

for-newcomers

An issue that should be newcomer friendly.

  

action

more-info-needed

More information is needed to investigate this issue

  

action

need-funding

This issue represents lots of work, we need to find money to pay our core team work

  

action

triage-required

It's not very clear what we want to do with this issue yet, more thoughts is required

  

kind

correctness

Issues of theoretical correctness that may impact edge-case scenarios

  

kind

ideas

For abstract ideas/proposal to make Garage better, to better communicate our not yet achieved aims

  

kind

improvement

Feature requests that do not fall in other categories

  

kind

performance

Deceptive performance observed OR idea to improve performances

  

kind

testing

Functional tests, integration tests, Jepsen, etc.

  

kind

usability

Propositions to make Garage easier to operate, by improving error reporting, CLI, etc.

  

kind

wrong-behavior

Unintended behavior

  

prio

critical

Could impact integrity or availability of production clusters

  

prio

low

For issues that we don't want to spend time on until made absolutely necessary

  

scope

admin-api

Issues concerning the administration API

  

scope

background-healing

Background healing covers repair tasks, anti-entropy, merkle trees, etc. ; everything related to replicating data appropriately.

  

scope

build

Issues with Nix, Drone, Rust Compiler, etc.

  

scope

documentation

  

scope

k8s

Kubernetes related, including Helm Charts

  

scope

layout

Linked to the cluster layout and storage management

  

scope

metadata

Metadata Engine (LMDB, sqlite, etc.)

  

scope

ops

Operations related to operations/maintenance issue, tooling related (like CLI missing info ; impossible, too dangerous, too complicated migration, etc.)

  

scope

rpc

Linked to RPC

  

scope

s3-api

Related to the S3 API (especially compatibility issues).

  

scope

security

Cryptography-related, auth-related, authorization-related issues

  

scope

telemetry

OpenTelemetry (traces, metrics, etc.)

No labels

action

check-aws

action

discussion-needed

action

for-external-contributors

action

for-newcomers

action

more-info-needed

action

need-funding

action

triage-required

kind

correctness

kind

ideas

kind

improvement

kind

performance

kind

testing

kind

usability

kind

wrong-behavior

prio

critical

prio

low

scope

admin-api

scope

background-healing

scope

build

scope

documentation

scope

k8s

scope

layout

scope

metadata

scope

ops

scope

rpc

scope

s3-api

scope

security

scope

telemetry

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Deuxfleurs/garage#298

No description provided.