Deuxfleurs
/
garage
43
37
Fork 55
Code Issues 69 Pull Requests 4 Releases 29 Wiki Activity

OpenAPI spec for admin API #379

Merged
lx merged 15 commits from ecosystem/openapi into main 2022-11-16 10:51:05 +00:00
Conversation 2 Commits 15 Files Changed 31 +3538 -684
quentin commented 2022-09-12 14:50:37 +00:00
Owner
Copy Link
  • last_seen_sec_ago can be null, cause an error with python currently
  • generated code could be way simpler and safer if we correctly tag mandatory fields
  • Add the spec for buckets, it is missing currently!
  • Document how to regenerate the API clients and the API doc
  • Regenerate the Python doc
  • Add examples for the bucket endpoints
  • Generate a Golang + Javascript API client
  • Update the doc to link to the new generated API doc, copy paste examples from the SDK README
  • Move the SDK repo to Deuxfleurs

Check:

- [X] `last_seen_sec_ago` can be null, cause an error with python currently - [X] generated code could be way simpler and safer if we correctly tag mandatory fields - [X] Add the spec for buckets, it is missing currently! - [X] Document how to regenerate the API clients and the API doc - [x] Regenerate the Python doc - [x] Add examples for the bucket endpoints - [x] Generate a Golang + Javascript API client - [x] Update the doc to link to the new generated API doc, copy paste examples from the SDK README - [x] Move the SDK repo to Deuxfleurs Check: - API DOC : https://garagehq.deuxfleurs.fr/api/garage-admin-v0.html - SDK : https://git.deuxfleurs.fr/garage-sdk/garage-admin-sdk-generator
quentin force-pushed ecosystem/openapi from 93ad9f09a5 to bbe38a9b30 2022-09-12 14:57:46 +00:00 Compare
lx changed title from WIP: OpenAPI spec to WIP: OpenAPI spec for admin API 2022-09-13 13:20:40 +00:00
quentin force-pushed ecosystem/openapi from 008119e1ba to ba67922310 2022-09-14 13:50:53 +00:00 Compare
quentin force-pushed ecosystem/openapi from 7b257b21c2 to 72a0f90070 2022-11-11 08:23:02 +00:00 Compare
quentin added 1 commit 2022-11-11 11:49:29 +00:00
continuous-integration/drone/push Build was killed Details
continuous-integration/drone/pr Build was killed Details
a976c9190c
Use awscli in the getting started guide
quentin added 1 commit 2022-11-11 15:57:23 +00:00
continuous-integration/drone/push Build was killed Details
continuous-integration/drone/pr Build was killed Details
dc50fa3b34
Fix typo in admin API on BucketInfo
quentin added 1 commit 2022-11-11 16:10:55 +00:00
continuous-integration/drone/push Build was killed Details
continuous-integration/drone/pr Build was killed Details
ebe8a41f2d
Bucket skeleton
quentin added 1 commit 2022-11-11 17:32:51 +00:00
continuous-integration/drone/push Build was killed Details
continuous-integration/drone/pr Build was killed Details
485109ea60
Bucket CRUD is defined
quentin added 1 commit 2022-11-12 17:08:57 +00:00
continuous-integration/drone/push Build was killed Details
continuous-integration/drone/pr Build was killed Details
e7824faa17
Finalize the specification of the admin API
quentin added 1 commit 2022-11-12 21:38:22 +00:00
continuous-integration/drone/push Build was killed Details
continuous-integration/drone/pr Build was killed Details
eabb37b53f
openapi validate fix
quentin added 1 commit 2022-11-12 22:05:02 +00:00
continuous-integration/drone/push Build was killed Details
continuous-integration/drone/pr Build was killed Details
74ea449f4b
Add missing parameter
quentin added 1 commit 2022-11-13 15:49:10 +00:00
continuous-integration/drone/pr Build is passing Details
continuous-integration/drone/push Build is passing Details
cf23aee183
Add a "build" section, doc for SDK
quentin changed title from WIP: OpenAPI spec for admin API to OpenAPI spec for admin API 2022-11-13 15:53:43 +00:00
lx commented 2022-11-14 12:05:56 +00:00
Owner
Copy Link

LGTM, awesome work.

Two questions:

  1. Do we really need to have all of these files in this repo ?
doc/api/css/redoc.css
doc/api/fonts/montserrat-v25-latin-300.woff
doc/api/fonts/montserrat-v25-latin-300.woff2
doc/api/fonts/montserrat-v25-latin-700.woff
doc/api/fonts/montserrat-v25-latin-700.woff2
doc/api/fonts/montserrat-v25-latin-regular.woff
doc/api/fonts/montserrat-v25-latin-regular.woff2
doc/api/fonts/roboto-v30-latin-300.woff
doc/api/fonts/roboto-v30-latin-300.woff2
doc/api/fonts/roboto-v30-latin-700.woff
doc/api/fonts/roboto-v30-latin-700.woff2
doc/api/fonts/roboto-v30-latin-regular.woff
doc/api/fonts/roboto-v30-latin-regular.woff2
doc/api/garage-admin-v0.html
doc/api/redoc.standalone.js

Couldn't we rather have them in the website repository, garagehq.deuxfleurs.fr ?

  1. This looks like it's going to be a lot of work to update the API specification file and regenerate everything when we will need to update the admin API. Can you talk about how you plan to manage this complexity? Do you commit to maintaining this part of the ecosystem in the future?
LGTM, awesome work. Two questions: 1. Do we really need to have all of these files in this repo ? ``` doc/api/css/redoc.css doc/api/fonts/montserrat-v25-latin-300.woff doc/api/fonts/montserrat-v25-latin-300.woff2 doc/api/fonts/montserrat-v25-latin-700.woff doc/api/fonts/montserrat-v25-latin-700.woff2 doc/api/fonts/montserrat-v25-latin-regular.woff doc/api/fonts/montserrat-v25-latin-regular.woff2 doc/api/fonts/roboto-v30-latin-300.woff doc/api/fonts/roboto-v30-latin-300.woff2 doc/api/fonts/roboto-v30-latin-700.woff doc/api/fonts/roboto-v30-latin-700.woff2 doc/api/fonts/roboto-v30-latin-regular.woff doc/api/fonts/roboto-v30-latin-regular.woff2 doc/api/garage-admin-v0.html doc/api/redoc.standalone.js ``` Couldn't we rather have them in the website repository, `garagehq.deuxfleurs.fr` ? 2. This looks like it's going to be a lot of work to update the API specification file and regenerate everything when we will need to update the admin API. Can you talk about how you plan to manage this complexity? Do you commit to maintaining this part of the ecosystem in the future?
quentin commented 2022-11-14 19:56:52 +00:00
Author
Owner
Copy Link

We can put the Redocly dependencies (fonts, css, js, etc.) in GarageHQ but we will loose a way to visualize and check the spec from Garage's repository. I have no strong opinion on that, feel free to do what you prefer here, we might find a tool that would be easy to install and that could be used to validate the file, without having to rely on this html+css+js viewer.

Concerning the API specification part, if we consider the API is designed to connect Garage to other software, we will have to maintain more or less stable interfaces over time, so I think clearly defining it is important. I would also prefer that we define the API first and then implement the code in a second time.

In practise, I just noticed that my specification is already outdated. I think that we can live with it as long as the change is backward compatible.

I also plan to use the Smithy IDL (Interface Definition Language) instead of OpenAPI which seems to be both less verbose, more versatile and more precise. If we want to reduce the complexity of maintaining both a specification and some codes, we could use some code generation tools (smithy can output an openapi spec than can be used to generate rust server code, or more directly, amazon has smithy-rs that can directly generate rust client+server code).

Directly writing our own Rust macros is also an option: Smithy can output machine readable JSON that could be used to generate our code for the JSON structures (aka the models in Smithy, schema in OpenAPI, and the serde strutures in our code) and the services endpoints, it would probably result in a more efficient and better tailored solution. I don't know if you want to follow this path, personally I don't find it necessary as long as we are able to stabilize APIs, but writing macros from the Smithy JSON output would be an interesting challenge ;-)

More generally, I think that my future work on Garage will be dedicated to create an ecosystem around it: I have the feeling that we have accomplished a lot and now, we must "unlock" this work to others by making it accessible/usable.

I would like to see Garage becoming an alternative to both Netlify/Vercel and Firebase/Firestore services. Given this goal, specifying clear interfaces on the K2V and administration API and having easy to use SDK for popular languages seems to be important. Being busy, I can't commit on being always available, but yes I think I can be listed as the maintainer of the Garage SDKs for now.

Edit: this talk might be of interest: https://www.youtube.com/watch?v=3GpZzu4guTE

We can put the Redocly dependencies (fonts, css, js, etc.) in GarageHQ but we will loose a way to visualize and check the spec from Garage's repository. I have no strong opinion on that, feel free to do what you prefer here, we might find a tool that would be easy to install and that could be used to validate the file, without having to rely on this html+css+js viewer. --- Concerning the API specification part, if we consider the API is designed to connect Garage to other software, we will have to maintain more or less stable interfaces over time, so I think clearly defining it is important. I would also prefer that we define the API first and then implement the code in a second time. In practise, I just noticed that my specification is already outdated. I think that we can live with it as long as the change is backward compatible. I also plan to use the [Smithy](https://smithy.io/2.0/index.html) IDL (Interface Definition Language) instead of OpenAPI which seems to be both less verbose, more versatile and more precise. If we want to reduce the complexity of maintaining both a specification and some codes, we could use some code generation tools (smithy can output an openapi spec than can be used [to generate rust server code](https://openapi-generator.tech/docs/generators/rust-server), or more directly, amazon has [smithy-rs](https://github.com/awslabs/smithy-rs) that can directly generate rust client+server code). Directly writing our own Rust macros is also an option: Smithy can output machine readable JSON that could be used to generate our code for the JSON structures (aka the models in Smithy, schema in OpenAPI, and the serde strutures in our code) and the services endpoints, it would probably result in a more efficient and better tailored solution. I don't know if you want to follow this path, personally I don't find it necessary as long as we are able to stabilize APIs, but writing macros from the Smithy JSON output would be an interesting challenge ;-) More generally, I think that my future work on Garage will be dedicated to create an ecosystem around it: I have the feeling that we have accomplished a lot and now, we must "unlock" this work to others by making it accessible/usable. I would like to see Garage becoming an alternative to both Netlify/Vercel and Firebase/Firestore services. Given this goal, specifying clear interfaces on the K2V and administration API and having easy to use SDK for popular languages seems to be important. Being busy, I can't commit on being always available, but yes I think I can be listed as the maintainer of the Garage SDKs for now. Edit: this talk might be of interest: https://www.youtube.com/watch?v=3GpZzu4guTE
👍 1
lx merged commit bcc9772470 into main 2022-11-16 10:51:05 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
AdminAPI

Issues concerning the administration API

  
Bug

   
Check AWS

The behaviour should be checked against AWS S3

  
CI

Issues with Nix, Drone, Rust Compiler, etc.

  
Correctness

Issues of theoretical correctness that may impact edge-case scenarios

  
Critical

   
Documentation

   
Ideas

For abstract ideas/proposal to make Garage better, to better communicate our not yet achieved aims

  
Improvement

   
Low priority

For issues that we don't want to spend time on until made absolutely necessary

  
Newcomer

A bug that should be newcomer friendly.

  
Performance

   
S3 Compatibility

   
Testing

   
Usability

Propositions to make Garage easier to operate, by improving error reporting, CLI, etc.

No Label
AdminAPI
Bug
Check AWS
CI
Correctness
Critical
Documentation
Ideas
Improvement
Low priority
Newcomer
Performance
S3 Compatibility
Testing
Usability
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: Deuxfleurs/garage#379
No description provided.
Delete Branch "ecosystem/openapi"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?