trinity-1686a/garage
2
0
Fork 0
forked from Deuxfleurs/garage
Code Issues Pull requests 1 Projects Releases Wiki Activity
1534 commits 13 branches 0 tags 42 MiB
Commit graph

301 commits

Author SHA1 Message Date
Max Audron 9d44127245
add support for kubernetes service discovery 
This commit adds support to discover garage instances running in
kubernetes.

Once enabled by setting `kubernetes_namespace` and
`kubernetes_service_name` garage will create a Custom Resources
`garagenodes.deuxfleurs.fr` with nodes public key as the resource name.
and IP and Port information as spec in the namespace configured by
`kubernetes_namespace`.

For discovering nodes the resources are filtered with the optionally set
`kubernetes_service_name` which sets a label
`garage.deuxfleurs.fr/service` on the resources.

This allows to separate multiple garage deployments in a single
namespace.

the `kubernetes_skip_crd` variable allows to disable the creation of the
CRD by garage itself. The user must deploy this manually.
 2022-03-12 13:05:52 +01:00
Alex e312ba977e
Add FOSDEM talk and move all .pdf files to Git LFS 2022-02-16 20:01:36 +01:00
Alex 2465163e39
documentation: add mention to install build-essential 2022-02-16 12:18:24 +01:00
Alex 4ae03aa774 Small documentation updates (#237) 
Fixes #234, among other things

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: Deuxfleurs/garage#237
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
 2022-02-10 15:58:09 +01:00
Quentin 3e1373fafc Add a new S3 comparison Matrix to documentation (#220) 
Co-authored-by: Quentin Dufour <quentin@deuxfleurs.fr>
Reviewed-on: Deuxfleurs/garage#220
Co-authored-by: Quentin <quentin@dufour.io>
Co-committed-by: Quentin <quentin@dufour.io>
 2022-02-07 16:04:52 +01:00
Alex 7d68b7060e
Fix anchors in links 2022-02-07 16:01:48 +01:00
Quentin 99ed67503c
Update quickstart to set endpoints ending in localhost 2022-02-07 16:01:48 +01:00
Quentin 5a1fb7cce7
Improve integration part of the doc 2022-02-07 16:01:45 +01:00
Alex 1c0ba930b8 Reorganize documentation for new website (#213) 
This PR should be merged after the new website is deployed.

- [x] Rename files
- [x] Add front matter section to all `.md` files in the book (necessary for Zola)
- [x] Change all internal links to use Zola's linking system that checks broken links
- [x] Some updates to documentation contents and organization

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: Deuxfleurs/garage#213
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
 2022-02-07 11:51:12 +01:00
Alex 2760f1cb17
Add advice about --fast-list 2022-01-31 16:51:39 +01:00
Alex 26849ed066
Add step to 0.6.0 migration guide 2022-01-27 14:31:25 +01:00
Alex ea7fb901eb
Implement {Put,Get,Delete}BucketCors and CORS in general 
- OPTIONS request against API endpoint
- Returning corresponding CORS headers on API calls
- Returning corresponding CORS headers on website GET's
 2022-01-24 11:58:00 +01:00
Quentin 94f0e7c135 Test ListParts endpoint with awscli 2022-01-21 10:42:35 +01:00
Alex 60c0033c8b
Update documentation 2022-01-13 14:25:22 +01:00
Alex 6617a72220
Implement UploadPartCopy 2022-01-13 13:58:47 +01:00
Quentin b4592a00fe Implement ListMultipartUploads (#171) 
Implement ListMultipartUploads, also refactor ListObjects and ListObjectsV2.

It took me some times as I wanted to propose the following things:
  - Using an iterator instead of the loop+goto pattern. I find it easier to read and it should enable some optimizations. For example, when consuming keys of a common prefix, we do many [redundant checks](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/src/api/s3_list.rs#L125-L156) while the only thing to do is to [check if the following key is still part of the common prefix](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/feature/s3-multipart-compat/src/api/s3_list.rs#L476).
  - Try to name things (see ExtractionResult and RangeBegin enums) and to separate concerns (see ListQuery and Accumulator)
  - An IO closure to make unit tests possibles.
  - Unit tests, to track regressions and document how to interact with the code
  - Integration tests with `s3api`. In the future, I would like to move them in Rust with the aws rust SDK.

Merging of the logic of ListMultipartUploads and ListObjects was not a goal but a consequence of the previous modifications.

Some points that we might want to discuss:
  - ListObjectsV1, when using pagination and delimiters, has a weird behavior (it lists multiple times the same prefix) with `aws s3api` due to the fact that it can not use our optimization to skip the whole prefix. It is independant from my refactor and can be tested with the commented `s3api` tests in `test-smoke.sh`. It probably has the same weird behavior on the official AWS S3 implementation.
  - Considering ListMultipartUploads, I had to "abuse" upload id marker to support prefix skipping. I send an `upload-id-marker` with the hardcoded value `include` to emulate your "including" token.
  - Some ways to test ListMultipartUploads with existing software (my tests are limited to s3api for now).

Co-authored-by: Quentin Dufour <quentin@deuxfleurs.fr>
Reviewed-on: Deuxfleurs/garage#171
Co-authored-by: Quentin <quentin@dufour.io>
Co-committed-by: Quentin <quentin@dufour.io>
 2022-01-12 19:04:55 +01:00
Alex 9cb2e9e57c
Add documentation to migrate to 0.6 2022-01-10 14:42:58 +01:00
Alex 8f39360f22
Update documentation 2022-01-07 17:14:37 +01:00
trinity-1686a 945b75dbf1 update s3 compatibility list (#177) 
Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com>
Reviewed-on: Deuxfleurs/garage#177
Co-authored-by: trinity-1686a <trinity.pointard@gmail.com>
Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
 2021-12-15 15:05:54 +01:00
trinity-1686a 1eb972b1ac Add compression using zstd (#173) 
fix #27

Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com>
Reviewed-on: Deuxfleurs/garage#173
Co-authored-by: trinity-1686a <trinity.pointard@gmail.com>
Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
 2021-12-15 11:26:43 +01:00
Quentin 3b3a1f275f
Add a second plot 2021-12-13 11:58:03 +01:00
Quentin dba9af2968
Update benchmark 2021-12-09 18:42:45 +01:00
Quentin e9358054ac
Typos and dead links 2021-12-08 14:40:14 +01:00
Quentin f9e5520ffb
Add a benchmark page with a first benchmark 2021-12-08 11:30:07 +01:00
Alex 4b369347c0
S3 compatibility target 2021-12-06 17:18:45 +01:00
Alex 224c89ad6e
Reorganize and improve documentation 2021-12-06 16:33:01 +01:00
Quentin 7c2037ba87
WIP front page garage 2021-12-06 15:33:43 +01:00
Alex c94406f428
Improve how node roles are assigned in Garage 
- change the terminology: the network configuration becomes the role
  table, the configuration of a nodes becomes a node's role
- the modification of the role table takes place in two steps: first,
  changes are staged in a CRDT data structure. Then, once the user is
  happy with the changes, they can commit them all at once (or revert
  them).
- update documentation
- fix tests
- implement smarter partition assignation algorithm

This patch breaks the format of the network configuration: when
migrating, the cluster will be in a state where no roles are assigned.
All roles must be re-assigned and commited at once. This migration
should not pose an issue.
 2021-11-16 16:05:53 +01:00
Trinity Pointard 53888995bd update doc and comments 2021-11-16 15:41:41 +01:00
Trinity Pointard 57df9c6e2d add s3_api.root_domain to doc book 2021-11-16 15:41:41 +01:00
adrien cdeb5b4dbb added link to RFID Garage talk (#155) 
Co-authored-by: ADRN <adrien@luxeylab.net>
Reviewed-on: Deuxfleurs/garage#155
Co-authored-by: adrien <adrien@luxeylab.net>
Co-committed-by: adrien <adrien@luxeylab.net>
 2021-11-16 15:39:08 +01:00
Quentin 100aad8bf4
Add rclone mount doc 2021-11-10 18:38:17 +01:00
Quentin 80a87929b0
Improve CLI documentation 2021-11-10 18:18:34 +01:00
Quentin 76d21be1b9
Add skeleton for backups, fuse and code sections 2021-11-10 18:05:07 +01:00
Quentin 1928f59d54
Add documentation for Gitea. 2021-11-10 12:41:09 +01:00
Quentin 323514be15
Documentation for Nix binary cache 2021-11-10 10:02:22 +01:00
Quentin ad8d5139cf
hugo deploy does not build website, fix doc 2021-11-10 10:02:22 +01:00
Alex 08b1e8a7ea
Move design draft to separate file; write about GC in internals 2021-11-09 12:25:33 +01:00
Quentin 8e25a37f0e
Add documentation for nginx 2021-11-08 12:20:40 +01:00
Quentin e342db19aa
Add documentation about Gateways 2021-11-08 12:20:40 +01:00
Quentin f3405b6378
Doc about exposing your website 2021-11-08 12:20:40 +01:00
Quentin 860ccf2811
Harden Garage's systemd service 2021-11-08 12:20:40 +01:00
Quentin 9df7559446
Documentation for hugo, jekyll and publii 2021-11-08 12:20:40 +01:00
Quentin a97467075d
Add documentation for synapse-s3-storage-provider 2021-11-08 12:20:40 +01:00
Trinity Pointard da6efb4b23 fix missing bootstrap_peers in doc 2021-11-05 11:21:50 +01:00
Quentin e93d7fb228
Add Peertube + improve CLI instructions 2021-11-03 14:39:14 +01:00
Quentin eaf54efb25
Add doc for Nextcloud 2021-11-03 14:07:55 +01:00
Quentin 93f8d59e4c
Extract toolchain build from the CI 2021-10-29 11:34:01 +02:00
ADRN 4e8af1d956 Modified the 'Funding' sentence to remove 'promise' since we actually got the first instalment 2021-10-26 13:34:28 +02:00
Alex 3e7f766d95
CLI: default rpc_host 2021-10-26 11:36:30 +02:00
Alex b2c51844a1
Add download link on homepage 2021-10-25 15:55:30 +02:00
Alex de4276202a
Improve CLI, adapt tests, update documentation 2021-10-25 14:21:48 +02:00
Quentin dc017a0cab
Build Garage with Nix 2021-10-19 16:56:07 +02:00
Alex 1aed317818
Small changes on NGI kickoff talk 2021-10-07 11:12:34 +02:00
Quentin c5574c8409 Add links and put logos in a flexbox 2021-09-28 10:21:10 +02:00
mricher de0228ca2a
Doc: add funding disclaimer for NGI/EU grant 
PNG logs optimized, render to be checked by @quentin. Fix #106.
 2021-09-25 17:21:07 +02:00
Alex df345e37db
Add sticker and NGI kickoff talk 2021-09-12 13:37:33 +02:00
Alex 42f692b1e0
Documentation fixes (typo and small reorganization) 2021-05-31 23:55:51 +02:00
Alex 14fd3df654
Write about S3 client configuration 2021-05-31 17:41:21 +02:00
Alex 56ac9fd460
Updates to documentation 2021-05-31 17:23:35 +02:00
Alex d76a8576f4
Reorganize documentation 2021-05-31 17:13:36 +02:00
Trinity Pointard 289521886b make most changes suggested during install-party 2021-05-29 21:37:49 +02:00
Alex ebd21b325e
Write documentation on configuration file and other improvements 2021-05-28 18:00:59 +02:00
Alex 339c611789
Add links to git/matrix/drone on top of documentation 2021-05-03 10:45:45 +02:00
Alex a981244f11
Replace talk links with permalinks 2021-04-30 15:51:32 +02:00
LUXEY Adrien be3b1d8f91 created doc/talks subfolder and added my talk and the previous one done at the wide team 2021-04-30 15:37:43 +02:00
Alex bf36f1f16a
Update documentation 2021-04-27 16:37:10 +02:00
Alex 9589d10165
fix command for adding node 2021-04-27 16:37:08 +02:00
Alex f9d77b6cd9
Section on recovering from failures 2021-04-27 16:37:08 +02:00
LUXEY Adrien ee00ac59b7
[doc] Added mention that GarageHQ is hosted with Garage 2021-04-27 16:37:08 +02:00
Alex c9943aab1f
fix typos 2021-04-27 16:37:07 +02:00
LUXEY Adrien d0692b75b0
intro.md: fix some typos, errors & dead links, plus some stylistic stuff 
modifié :         doc/book/src/intro.md
 2021-04-27 16:37:07 +02:00
Quentin 4a6ed223dc
Fix garage_util description 2021-04-27 16:37:07 +02:00
Quentin c409ee89f6
Fix some typos 2021-04-27 16:37:07 +02:00
Quentin c1d64333c5
Fix a table in the doc 2021-04-27 16:37:04 +02:00
Alex c5e24de159 Center logo in book, add book CI 2021-03-19 14:25:57 +01:00
Alex 4348bde180 Merge branch 'dev-0.2' 2021-03-18 19:27:02 +01:00
Quentin ea21c54434 Add handle files section to the doc 2021-03-17 22:44:35 +01:00
Quentin 1a5af9d1fc WIP getting started 2021-03-17 22:06:37 +01:00
Quentin b82a61fba2 Simplify our README 2021-03-17 20:58:30 +01:00
Quentin 44d0815ff9 Wrote daemon 2021-03-17 20:04:27 +01:00
Quentin 468e45ed7f WIP doc 2021-03-17 18:01:06 +01:00
Quentin 60f994a118 Working on the getting started guide 2021-03-17 17:24:11 +01:00
Quentin 002538f92c Refactor file organization 2021-03-17 16:15:18 +01:00
Quentin c50113acf3 Work on structure + Getting started is reworked 2021-03-17 15:44:29 +01:00
Quentin 0afc701a69 Doc skeleton + intro 2021-03-17 14:44:14 +01:00
Alex 797cda1c33 Add logo in repo 2021-03-17 10:26:50 +01:00
Alex 46c7226fe4 New logo!! 2021-03-17 08:41:07 +01:00
Alex d7e148d302 Description of MultiDC MagLev 2021-02-25 11:37:42 +01:00
Alex 0522983aec precisions 2021-02-25 11:27:13 +01:00
Alex fdf908e845 Add precisions 2021-02-25 11:23:22 +01:00
Alex 2b4b69938f Add write-up about load-balancing 2021-02-25 11:18:44 +01:00
Alex 0ddfee92c5 add precision 2021-02-19 19:11:55 +01:00
Alex 5d1fa591d9 Add compatibility list 2021-02-19 19:10:23 +01:00
Alex f26c795c99 Add warning 2021-01-23 19:15:57 +01:00
Alex 6f6bf23bec Move doc files here from Deuxfleur's site repo 2021-01-23 19:12:10 +01:00
Alex f844d4ee9b Add slide on consistency 2020-12-01 17:42:13 +01:00
Alex 7642229d54 Two new slides 2020-12-01 14:31:13 +01:00
Alex ad432eb154 Add some first technical slides 2020-12-01 13:46:23 +01:00
Quentin 9a57a0319a Talk 2020-11-30 17:32:37 +01:00
Alex 9b3aabfcbf Add talk template 2020-11-30 15:54:31 +01:00