203 commits

Author	SHA1	Message	Date
Alex	b74b533b7b	Fix typo	2022-06-29 11:50:51 +02:00
Alex	77e3fd6db2	improve internal item counter mechanisms and implement bucket quotas (#326) ... - [x] Refactoring of internal counting API - [x] Repair procedure for counters (it's an offline procedure!!!) - [x] New counter for objects in buckets - [x] Add quotas to buckets struct - [x] Add CLI to manage bucket quotas - [x] Add admin API to manage bucket quotas - [x] Apply quotas by adding checks on put operations - [x] Proof-read Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: #326 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>	2022-06-15 20:20:28 +02:00
Simon C	7eed3ceda9	docs: Add Trafik reverse proxy documentation	2022-06-07 16:16:52 +02:00
Simon C	4b8f48f3c5	docs: Fix title level	2022-06-07 13:32:52 +02:00
Simon C	7d3b5585f1	docs: Add link to facilitate navigation in the documentation	2022-06-07 09:38:59 +02:00
Quentin Dufour	a1abed0378	Remove useless MC_REGION env variable	2022-06-02 12:50:11 +02:00
Alex	93eab8eaa3	Fixes to S3 compatibility page (#314) ... Mention PostObject is implemented, fix english mistakes Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: #314 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>	2022-05-25 16:54:44 +02:00
Quentin Dufour	43ddc933f9	Update Ceph S3 endpoints compatibility	2022-05-25 15:20:08 +02:00
Alex Auvolat	9f303f6308	Shorter page title	2022-05-24 15:47:42 +02:00
Alex Auvolat	3be43f3372	Add lost content for Restic with Garage ... Suggested-by: Quentin <quentin@deuxfleurs.fr>	2022-05-24 15:32:42 +02:00
Alex Auvolat	2da448b43f	Add documentation for new Admin API and a few infos on K2V	2022-05-24 15:28:37 +02:00
Baptiste Jonglez	47e57518ec	Add documentation on running Kopia with Garage	2022-04-10 13:04:07 +02:00
Quentin Dufour	8e3ee82c3e	Be clearer on what upgrades are (not) supported	2022-04-06 21:45:59 +02:00
Quentin Dufour	a122a8cb46	Add an "upgrading" section, add a guide for 0.7	2022-04-05 10:08:31 +02:00
Quentin Dufour	9fd8ec1dee	Add documentation for winscp+sftpgo	2022-03-31 10:25:56 +02:00
Alex Auvolat	0091002ef2	New replication modes and their documentation	2022-03-28 16:26:04 +02:00
Alex Auvolat	dca2ffdf91	document administrative options	2022-03-28 12:26:08 +02:00
Alex Auvolat	822128e3c8	Talk a bit about capacity balancing between regions	2022-03-22 12:07:13 +01:00
Rune Henriksen	aea8b41728	document request routing logic	2022-03-21 12:03:57 +01:00
Rune Henriksen	71e6645e09	add short tutorial for duplicati usage with garage	2022-03-21 11:58:19 +01:00
Steam	15da2156f6	Change position of the node-id argument	2022-03-19 18:03:23 +01:00
Max Audron	9d44127245	add support for kubernetes service discovery ... This commit adds support to discover garage instances running in kubernetes. Once enabled by setting `kubernetes_namespace` and `kubernetes_service_name` garage will create a Custom Resources `garagenodes.deuxfleurs.fr` with nodes public key as the resource name. and IP and Port information as spec in the namespace configured by `kubernetes_namespace`. For discovering nodes the resources are filtered with the optionally set `kubernetes_service_name` which sets a label `garage.deuxfleurs.fr/service` on the resources. This allows to separate multiple garage deployments in a single namespace. the `kubernetes_skip_crd` variable allows to disable the creation of the CRD by garage itself. The user must deploy this manually.	2022-03-12 13:05:52 +01:00
Alex Auvolat	2465163e39	documentation: add mention to install build-essential	2022-02-16 12:18:24 +01:00
Alex	4ae03aa774	Small documentation updates (#237) ... Fixes #234, among other things Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: #237 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>	2022-02-10 15:58:09 +01:00
Quentin	3e1373fafc	Add a new S3 comparison Matrix to documentation (#220) ... Co-authored-by: Quentin Dufour <quentin@deuxfleurs.fr> Reviewed-on: #220 Co-authored-by: Quentin <quentin@dufour.io> Co-committed-by: Quentin <quentin@dufour.io>	2022-02-07 16:04:52 +01:00
Alex Auvolat	7d68b7060e	Fix anchors in links	2022-02-07 16:01:48 +01:00
Quentin Dufour	99ed67503c	Update quickstart to set endpoints ending in localhost	2022-02-07 16:01:48 +01:00
Quentin Dufour	5a1fb7cce7	Improve integration part of the doc	2022-02-07 16:01:45 +01:00
Alex	1c0ba930b8	Reorganize documentation for new website (#213) ... This PR should be merged after the new website is deployed. - [x] Rename files - [x] Add front matter section to all `.md` files in the book (necessary for Zola) - [x] Change all internal links to use Zola's linking system that checks broken links - [x] Some updates to documentation contents and organization Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: #213 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>	2022-02-07 11:51:12 +01:00
Alex Auvolat	2760f1cb17	Add advice about --fast-list	2022-01-31 16:51:39 +01:00
Alex Auvolat	26849ed066	Add step to 0.6.0 migration guide	2022-01-27 14:31:25 +01:00
Alex Auvolat	ea7fb901eb	Implement {Put,Get,Delete}BucketCors and CORS in general ... - OPTIONS request against API endpoint - Returning corresponding CORS headers on API calls - Returning corresponding CORS headers on website GET's	2022-01-24 11:58:00 +01:00
Quentin Dufour	94f0e7c135	Test ListParts endpoint with awscli	2022-01-21 10:42:35 +01:00
Alex Auvolat	60c0033c8b	Update documentation	2022-01-13 14:25:22 +01:00
Alex Auvolat	6617a72220	Implement UploadPartCopy	2022-01-13 13:58:47 +01:00
Quentin	b4592a00fe	Implement ListMultipartUploads (#171) ... Implement ListMultipartUploads, also refactor ListObjects and ListObjectsV2. It took me some times as I wanted to propose the following things: - Using an iterator instead of the loop+goto pattern. I find it easier to read and it should enable some optimizations. For example, when consuming keys of a common prefix, we do many [redundant checks](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/src/api/s3_list.rs#L125-L156) while the only thing to do is to [check if the following key is still part of the common prefix](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/feature/s3-multipart-compat/src/api/s3_list.rs#L476). - Try to name things (see ExtractionResult and RangeBegin enums) and to separate concerns (see ListQuery and Accumulator) - An IO closure to make unit tests possibles. - Unit tests, to track regressions and document how to interact with the code - Integration tests with `s3api`. In the future, I would like to move them in Rust with the aws rust SDK. Merging of the logic of ListMultipartUploads and ListObjects was not a goal but a consequence of the previous modifications. Some points that we might want to discuss: - ListObjectsV1, when using pagination and delimiters, has a weird behavior (it lists multiple times the same prefix) with `aws s3api` due to the fact that it can not use our optimization to skip the whole prefix. It is independant from my refactor and can be tested with the commented `s3api` tests in `test-smoke.sh`. It probably has the same weird behavior on the official AWS S3 implementation. - Considering ListMultipartUploads, I had to "abuse" upload id marker to support prefix skipping. I send an `upload-id-marker` with the hardcoded value `include` to emulate your "including" token. - Some ways to test ListMultipartUploads with existing software (my tests are limited to s3api for now). Co-authored-by: Quentin Dufour <quentin@deuxfleurs.fr> Reviewed-on: #171 Co-authored-by: Quentin <quentin@dufour.io> Co-committed-by: Quentin <quentin@dufour.io>	2022-01-12 19:04:55 +01:00
Alex Auvolat	9cb2e9e57c	Add documentation to migrate to 0.6	2022-01-10 14:42:58 +01:00
Alex Auvolat	8f39360f22	Update documentation	2022-01-07 17:14:37 +01:00
trinity-1686a	945b75dbf1	update s3 compatibility list (#177) ... Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com> Reviewed-on: #177 Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>	2021-12-15 15:05:54 +01:00
trinity-1686a	1eb972b1ac	Add compression using zstd (#173) ... fix #27 Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com> Reviewed-on: #173 Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>	2021-12-15 11:26:43 +01:00
Quentin Dufour	3b3a1f275f	Add a second plot	2021-12-13 11:58:03 +01:00
Quentin Dufour	dba9af2968	Update benchmark	2021-12-09 18:42:45 +01:00
Quentin Dufour	e9358054ac	Typos and dead links	2021-12-08 14:40:14 +01:00
Quentin Dufour	f9e5520ffb	Add a benchmark page with a first benchmark	2021-12-08 11:30:07 +01:00
Alex Auvolat	4b369347c0	S3 compatibility target	2021-12-06 17:18:45 +01:00
Alex Auvolat	224c89ad6e	Reorganize and improve documentation	2021-12-06 16:33:01 +01:00
Quentin Dufour	7c2037ba87	WIP front page garage	2021-12-06 15:33:43 +01:00
Alex Auvolat	c94406f428	Improve how node roles are assigned in Garage ... - change the terminology: the network configuration becomes the role table, the configuration of a nodes becomes a node's role - the modification of the role table takes place in two steps: first, changes are staged in a CRDT data structure. Then, once the user is happy with the changes, they can commit them all at once (or revert them). - update documentation - fix tests - implement smarter partition assignation algorithm This patch breaks the format of the network configuration: when migrating, the cluster will be in a state where no roles are assigned. All roles must be re-assigned and commited at once. This migration should not pose an issue.	2021-11-16 16:05:53 +01:00
Trinity Pointard	53888995bd	update doc and comments	2021-11-16 15:41:41 +01:00
Trinity Pointard	57df9c6e2d	add s3_api.root_domain to doc book	2021-11-16 15:41:41 +01:00
adrien	cdeb5b4dbb	added link to RFID Garage talk (#155) ... Co-authored-by: ADRN <adrien@luxeylab.net> Reviewed-on: #155 Co-authored-by: adrien <adrien@luxeylab.net> Co-committed-by: adrien <adrien@luxeylab.net>	2021-11-16 15:39:08 +01:00
Quentin Dufour	100aad8bf4	Add rclone mount doc	2021-11-10 18:38:17 +01:00
Quentin Dufour	80a87929b0	Improve CLI documentation	2021-11-10 18:18:34 +01:00
Quentin Dufour	76d21be1b9	Add skeleton for backups, fuse and code sections	2021-11-10 18:05:07 +01:00
Quentin Dufour	1928f59d54	Add documentation for Gitea.	2021-11-10 12:41:09 +01:00
Quentin Dufour	323514be15	Documentation for Nix binary cache	2021-11-10 10:02:22 +01:00
Quentin Dufour	ad8d5139cf	hugo deploy does not build website, fix doc	2021-11-10 10:02:22 +01:00
Alex Auvolat	08b1e8a7ea	Move design draft to separate file; write about GC in internals	2021-11-09 12:25:33 +01:00
Quentin Dufour	8e25a37f0e	Add documentation for nginx	2021-11-08 12:20:40 +01:00
Quentin Dufour	e342db19aa	Add documentation about Gateways	2021-11-08 12:20:40 +01:00
Quentin Dufour	f3405b6378	Doc about exposing your website	2021-11-08 12:20:40 +01:00
Quentin Dufour	860ccf2811	Harden Garage's systemd service	2021-11-08 12:20:40 +01:00
Quentin Dufour	9df7559446	Documentation for hugo, jekyll and publii	2021-11-08 12:20:40 +01:00
Quentin Dufour	a97467075d	Add documentation for synapse-s3-storage-provider	2021-11-08 12:20:40 +01:00
Trinity Pointard	da6efb4b23	fix missing bootstrap_peers in doc	2021-11-05 11:21:50 +01:00
Quentin Dufour	e93d7fb228	Add Peertube + improve CLI instructions	2021-11-03 14:39:14 +01:00
Quentin Dufour	eaf54efb25	Add doc for Nextcloud	2021-11-03 14:07:55 +01:00
Quentin Dufour	93f8d59e4c	Extract toolchain build from the CI	2021-10-29 11:34:01 +02:00
ADRN	4e8af1d956	Modified the 'Funding' sentence to remove 'promise' since we actually got the first instalment	2021-10-26 13:34:28 +02:00
Alex Auvolat	3e7f766d95	CLI: default rpc_host	2021-10-26 11:36:30 +02:00
Alex Auvolat	b2c51844a1	Add download link on homepage	2021-10-25 15:55:30 +02:00
Alex Auvolat	de4276202a	Improve CLI, adapt tests, update documentation	2021-10-25 14:21:48 +02:00
Quentin Dufour	dc017a0cab	Build Garage with Nix	2021-10-19 16:56:07 +02:00
Quentin	c5574c8409	Add links and put logos in a flexbox	2021-09-28 10:21:10 +02:00
mricher	de0228ca2a	Doc: add funding disclaimer for NGI/EU grant ... PNG logs optimized, render to be checked by @quentin. Fix #106.	2021-09-25 17:21:07 +02:00
Alex Auvolat	42f692b1e0	Documentation fixes (typo and small reorganization)	2021-05-31 23:55:51 +02:00
Alex Auvolat	14fd3df654	Write about S3 client configuration	2021-05-31 17:41:21 +02:00
Alex Auvolat	56ac9fd460	Updates to documentation	2021-05-31 17:23:35 +02:00
Alex Auvolat	d76a8576f4	Reorganize documentation	2021-05-31 17:13:36 +02:00
Trinity Pointard	289521886b	make most changes suggested during install-party	2021-05-29 21:37:49 +02:00
Alex Auvolat	ebd21b325e	Write documentation on configuration file and other improvements	2021-05-28 18:00:59 +02:00
Alex Auvolat	339c611789	Add links to git/matrix/drone on top of documentation	2021-05-03 10:45:45 +02:00
Alex Auvolat	a981244f11	Replace talk links with permalinks	2021-04-30 15:51:32 +02:00
LUXEY Adrien	be3b1d8f91	created doc/talks subfolder and added my talk and the previous one done at the wide team	2021-04-30 15:37:43 +02:00
Alex Auvolat	bf36f1f16a	Update documentation	2021-04-27 16:37:10 +02:00
Alex Auvolat	9589d10165	fix command for adding node	2021-04-27 16:37:08 +02:00
Alex Auvolat	f9d77b6cd9	Section on recovering from failures	2021-04-27 16:37:08 +02:00
LUXEY Adrien	ee00ac59b7	[doc] Added mention that GarageHQ is hosted with Garage	2021-04-27 16:37:08 +02:00
Alex Auvolat	c9943aab1f	fix typos	2021-04-27 16:37:07 +02:00
LUXEY Adrien	d0692b75b0	intro.md: fix some typos, errors & dead links, plus some stylistic stuff ... modifié : doc/book/src/intro.md	2021-04-27 16:37:07 +02:00
Quentin Dufour	4a6ed223dc	Fix garage_util description	2021-04-27 16:37:07 +02:00
Quentin Dufour	c409ee89f6	Fix some typos	2021-04-27 16:37:07 +02:00
Quentin Dufour	c1d64333c5	Fix a table in the doc	2021-04-27 16:37:04 +02:00
Alex Auvolat	c5e24de159	Center logo in book, add book CI	2021-03-19 14:25:57 +01:00
Quentin Dufour	ea21c54434	Add handle files section to the doc	2021-03-17 22:44:35 +01:00
Quentin Dufour	1a5af9d1fc	WIP getting started	2021-03-17 22:06:37 +01:00
Quentin Dufour	b82a61fba2	Simplify our README	2021-03-17 20:58:30 +01:00
Quentin Dufour	44d0815ff9	Wrote daemon	2021-03-17 20:04:27 +01:00
Quentin Dufour	468e45ed7f	WIP doc	2021-03-17 18:01:06 +01:00
Quentin Dufour	60f994a118	Working on the getting started guide	2021-03-17 17:24:11 +01:00
Quentin Dufour	002538f92c	Refactor file organization	2021-03-17 16:15:18 +01:00
Quentin Dufour	c50113acf3	Work on structure + Getting started is reworked	2021-03-17 15:44:29 +01:00
Quentin Dufour	0afc701a69	Doc skeleton + intro	2021-03-17 14:44:14 +01:00