205 commits

Author	SHA1	Message	Date
Alex	c2a9f00a58	Merge pull request 'upgrading.md: Added small note about garage_build_info.' (#501) from jpds/garage:doc-upgrade-buildinfo-metric into main ... Reviewed-on: #501	2023-02-06 14:20:00 +00:00
Jonathan Davies	179fda9fb6	upgrading.md: Added small note about garage_build_info.	2023-02-06 12:53:55 +00:00
Alex Auvolat	80e2326998	fixes for pr 499	2023-02-06 12:23:55 +01:00
Alex Auvolat	656b8d42de	secrets can be passed directly in config, as file, or as env	2023-02-03 15:27:39 +01:00
Alex Auvolat	8013a5cd58	Change talk links more	2023-01-30 18:51:48 +01:00
Alex Auvolat	2ba9463a8a	Raw links to presentations	2023-01-30 18:48:00 +01:00
Alex Auvolat	7f715ba94f	zero-downtime migration procedure	2023-01-30 18:41:04 +01:00
Alex Auvolat	44f8b1d71a	Reorder reference manual section, move metrics list to there	2023-01-30 18:00:01 +01:00
Alex Auvolat	56384677fa	Add links to presentations	2023-01-30 17:48:36 +01:00
Jonathan Davies	5f412abd4e	cookbook/reverse-proxy.md: Added on-demand TLS section.	2023-01-30 14:37:55 +00:00
Jonathan Davies	c753a9dfb6	cookbook/monitoring.md: Added new metrics (garage_build_info, ... garage_replication_factor, block_compression_level).	2023-01-30 12:54:42 +00:00
Jonathan Davies	ae9c7a2900	cookbook/_index.md: Added link to monitoring documentation.	2023-01-30 12:54:42 +00:00
Jonathan Davies	7ab27f84b8	configuration.md: Corrected OpenTelemetry.	2023-01-30 12:54:42 +00:00
Jonathan Davies	55c369137d	gateways.md: -z is a required flag for layout assign.	2023-01-30 12:54:38 +00:00
Jonathan Davies	0c618f8a89	reverse-proxy.md: Corrected web server ports in Caddy example.	2023-01-27 17:52:51 +00:00
Alex	a08e01f17a	Merge pull request 'Enable daemonset deployment using the helm chart' (#409) from kaiyou/garage:feat-k8s-daemonset into main ... Reviewed-on: #409	2023-01-26 21:07:58 +00:00
Jonathan Davies	f2492107d7	cookbook/real-world.md: Added note about mesh network options.	2023-01-25 12:00:01 +00:00
Jonathan Davies	97bb110219	doc: Added observability.md.	2023-01-13 14:32:10 +00:00
kaiyou	9bf94faaa1	Add docs about running pict-rs with garage	2023-01-12 20:46:17 +01:00
Felix Scheinost	f2106c2733	Implement rpc_secret_file	2023-01-04 18:35:10 +01:00
Jonathan Davies	8be862aa19	Changed all instances of 'key new' to 'key create' to make it consistent as bucket commands issued normally around the same time.	2023-01-03 11:11:12 +00:00
Alex	1649002e2b	Merge pull request 'Add a note about Peertube 5.0 private videos' (#456) from kaiyou/garage:docs-apps into main ... Reviewed-on: #456	2023-01-02 12:49:14 +00:00
kaiyou	fd10200bec	Add a note about Peertube 5.0 private videos	2022-12-25 14:20:01 +01:00
kaiyou	0c7ed0b0af	Add some docs about using Python Minio SDK	2022-12-25 13:55:12 +01:00
kaiyou	e852c91d18	Fix documentation based on new deployment values	2022-12-25 13:30:14 +01:00
felix.scheinost	d5a2502b09	Fix typo in documentation	2022-12-07 12:43:49 +00:00
tompearson	a3afc761b6	Update 'doc/book/design/goals.md'	2022-12-04 16:27:46 +00:00
Alex Auvolat	379b2049f5	Fix #432: documentation issue	2022-11-24 15:33:33 +01:00
Alex Auvolat	555a54ec40	doc precisions and fixes	2022-11-16 13:40:49 +01:00
Alex Auvolat	fc8f795bba	Rename subsections and add docker compose file	2022-11-16 13:33:33 +01:00
Alex Auvolat	a7af0c8af9	Add best practices and doc of monitoring (fix #419)	2022-11-16 13:27:24 +01:00
Alex	bcc9772470	Merge pull request 'OpenAPI spec for admin API' (#379) from ecosystem/openapi into main ... Reviewed-on: #379	2022-11-16 10:51:04 +00:00
Alex Auvolat	05547f2ba6	Move testing strategy to a dedicated doc section (fix #114)	2022-11-14 13:34:00 +01:00
Quentin Dufour	cf23aee183	Add a "build" section, doc for SDK	2022-11-13 16:48:52 +01:00
Baptiste Jonglez	8dfc909759	Improve Nginx reverse proxy example ... By default, Nginx does proxy buffering and it may store big replies to a temporary file up to 1 GB. It also means that Nginx will read data as fast as possible from Garage, even if the client downloads slowly. Both behaviours are often not wanted, so disable this temporary file in the example. Ref: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering Also add an example of upstream with a "backup" server, which may be useful to only use remote servers as fallback.	2022-11-11 21:50:08 +01:00
Quentin Dufour	a976c9190c	Use awscli in the getting started guide	2022-11-11 12:48:52 +01:00
Baptiste Jonglez	26b3295aaa	Add documentation to run Mastodon on Garage	2022-11-06 14:07:31 +01:00
Alex Auvolat	8bc5caf7aa	Fix issue with 'http(s)://' prefix	2022-10-18 21:17:11 +02:00
Alex Auvolat	002b9fc50c	Add TLS support for Consul discovery + refactoring	2022-10-18 18:38:20 +02:00
borgified	8e442001b9	Update 'doc/book/reference-manual/features.md' ... typo	2022-10-16 07:13:21 +00:00
chemicstry	d2c937a931	Fix typo	2022-09-30 18:46:57 +02:00
chemicstry	744c3b4d94	Update docs	2022-09-30 18:46:57 +02:00
chemicstry	37a73d7d37	Move documentation to book	2022-09-30 18:46:57 +02:00
Alex Auvolat	0ab0d3cc29	Document db_engine	2022-09-27 16:52:36 +02:00
Alex	7291747a28	Merge pull request 'Documentation changes for v0.8' (#394) from doc-0.8 into main ... Reviewed-on: #394	2022-09-27 11:37:12 +02:00
Alex Auvolat	194e8be1bb	Update docker image links	2022-09-26 18:01:17 +02:00
Alex Auvolat	69bcc813de	Add garage v0.8 migration guide	2022-09-26 17:46:38 +02:00
Alex Auvolat	a3758dc4c4	Update README	2022-09-21 12:53:02 +02:00
Alex Auvolat	1d0a610690	Finish writing about Garage features, and fix from-source instructions	2022-09-15 13:23:57 +02:00
Alex Auvolat	f6aebefcc9	Some work on documentation towards v0.8	2022-09-14 19:31:13 +02:00
Alex Auvolat	bbb970965c	Document available build features	2022-09-06 17:16:45 +02:00
Quentin Dufour	532eca7ff9	Add some documentation for Caddy	2022-08-12 10:33:41 +02:00
Alex	b74b533b7b	Fix typo	2022-06-29 11:50:51 +02:00
Alex	77e3fd6db2	improve internal item counter mechanisms and implement bucket quotas (#326) ... - [x] Refactoring of internal counting API - [x] Repair procedure for counters (it's an offline procedure!!!) - [x] New counter for objects in buckets - [x] Add quotas to buckets struct - [x] Add CLI to manage bucket quotas - [x] Add admin API to manage bucket quotas - [x] Apply quotas by adding checks on put operations - [x] Proof-read Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: #326 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>	2022-06-15 20:20:28 +02:00
Simon C	7eed3ceda9	docs: Add Trafik reverse proxy documentation	2022-06-07 16:16:52 +02:00
Simon C	4b8f48f3c5	docs: Fix title level	2022-06-07 13:32:52 +02:00
Simon C	7d3b5585f1	docs: Add link to facilitate navigation in the documentation	2022-06-07 09:38:59 +02:00
Quentin Dufour	a1abed0378	Remove useless MC_REGION env variable	2022-06-02 12:50:11 +02:00
Alex	93eab8eaa3	Fixes to S3 compatibility page (#314) ... Mention PostObject is implemented, fix english mistakes Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: #314 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>	2022-05-25 16:54:44 +02:00
Quentin Dufour	43ddc933f9	Update Ceph S3 endpoints compatibility	2022-05-25 15:20:08 +02:00
Alex Auvolat	9f303f6308	Shorter page title	2022-05-24 15:47:42 +02:00
Alex Auvolat	3be43f3372	Add lost content for Restic with Garage ... Suggested-by: Quentin <quentin@deuxfleurs.fr>	2022-05-24 15:32:42 +02:00
Alex Auvolat	2da448b43f	Add documentation for new Admin API and a few infos on K2V	2022-05-24 15:28:37 +02:00
Baptiste Jonglez	47e57518ec	Add documentation on running Kopia with Garage	2022-04-10 13:04:07 +02:00
Quentin Dufour	8e3ee82c3e	Be clearer on what upgrades are (not) supported	2022-04-06 21:45:59 +02:00
Quentin Dufour	a122a8cb46	Add an "upgrading" section, add a guide for 0.7	2022-04-05 10:08:31 +02:00
Quentin Dufour	9fd8ec1dee	Add documentation for winscp+sftpgo	2022-03-31 10:25:56 +02:00
Alex Auvolat	0091002ef2	New replication modes and their documentation	2022-03-28 16:26:04 +02:00
Alex Auvolat	dca2ffdf91	document administrative options	2022-03-28 12:26:08 +02:00
Alex Auvolat	822128e3c8	Talk a bit about capacity balancing between regions	2022-03-22 12:07:13 +01:00
Rune Henriksen	aea8b41728	document request routing logic	2022-03-21 12:03:57 +01:00
Rune Henriksen	71e6645e09	add short tutorial for duplicati usage with garage	2022-03-21 11:58:19 +01:00
Steam	15da2156f6	Change position of the node-id argument	2022-03-19 18:03:23 +01:00
Max Audron	9d44127245	add support for kubernetes service discovery ... This commit adds support to discover garage instances running in kubernetes. Once enabled by setting `kubernetes_namespace` and `kubernetes_service_name` garage will create a Custom Resources `garagenodes.deuxfleurs.fr` with nodes public key as the resource name. and IP and Port information as spec in the namespace configured by `kubernetes_namespace`. For discovering nodes the resources are filtered with the optionally set `kubernetes_service_name` which sets a label `garage.deuxfleurs.fr/service` on the resources. This allows to separate multiple garage deployments in a single namespace. the `kubernetes_skip_crd` variable allows to disable the creation of the CRD by garage itself. The user must deploy this manually.	2022-03-12 13:05:52 +01:00
Alex Auvolat	2465163e39	documentation: add mention to install build-essential	2022-02-16 12:18:24 +01:00
Alex	4ae03aa774	Small documentation updates (#237) ... Fixes #234, among other things Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: #237 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>	2022-02-10 15:58:09 +01:00
Quentin	3e1373fafc	Add a new S3 comparison Matrix to documentation (#220) ... Co-authored-by: Quentin Dufour <quentin@deuxfleurs.fr> Reviewed-on: #220 Co-authored-by: Quentin <quentin@dufour.io> Co-committed-by: Quentin <quentin@dufour.io>	2022-02-07 16:04:52 +01:00
Alex Auvolat	7d68b7060e	Fix anchors in links	2022-02-07 16:01:48 +01:00
Quentin Dufour	99ed67503c	Update quickstart to set endpoints ending in localhost	2022-02-07 16:01:48 +01:00
Quentin Dufour	5a1fb7cce7	Improve integration part of the doc	2022-02-07 16:01:45 +01:00
Alex	1c0ba930b8	Reorganize documentation for new website (#213) ... This PR should be merged after the new website is deployed. - [x] Rename files - [x] Add front matter section to all `.md` files in the book (necessary for Zola) - [x] Change all internal links to use Zola's linking system that checks broken links - [x] Some updates to documentation contents and organization Co-authored-by: Alex Auvolat <alex@adnab.me> Reviewed-on: #213 Co-authored-by: Alex <alex@adnab.me> Co-committed-by: Alex <alex@adnab.me>	2022-02-07 11:51:12 +01:00
Alex Auvolat	2760f1cb17	Add advice about --fast-list	2022-01-31 16:51:39 +01:00
Alex Auvolat	26849ed066	Add step to 0.6.0 migration guide	2022-01-27 14:31:25 +01:00
Alex Auvolat	ea7fb901eb	Implement {Put,Get,Delete}BucketCors and CORS in general ... - OPTIONS request against API endpoint - Returning corresponding CORS headers on API calls - Returning corresponding CORS headers on website GET's	2022-01-24 11:58:00 +01:00
Quentin Dufour	94f0e7c135	Test ListParts endpoint with awscli	2022-01-21 10:42:35 +01:00
Alex Auvolat	60c0033c8b	Update documentation	2022-01-13 14:25:22 +01:00
Alex Auvolat	6617a72220	Implement UploadPartCopy	2022-01-13 13:58:47 +01:00
Quentin	b4592a00fe	Implement ListMultipartUploads (#171) ... Implement ListMultipartUploads, also refactor ListObjects and ListObjectsV2. It took me some times as I wanted to propose the following things: - Using an iterator instead of the loop+goto pattern. I find it easier to read and it should enable some optimizations. For example, when consuming keys of a common prefix, we do many [redundant checks](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/src/api/s3_list.rs#L125-L156) while the only thing to do is to [check if the following key is still part of the common prefix](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/feature/s3-multipart-compat/src/api/s3_list.rs#L476). - Try to name things (see ExtractionResult and RangeBegin enums) and to separate concerns (see ListQuery and Accumulator) - An IO closure to make unit tests possibles. - Unit tests, to track regressions and document how to interact with the code - Integration tests with `s3api`. In the future, I would like to move them in Rust with the aws rust SDK. Merging of the logic of ListMultipartUploads and ListObjects was not a goal but a consequence of the previous modifications. Some points that we might want to discuss: - ListObjectsV1, when using pagination and delimiters, has a weird behavior (it lists multiple times the same prefix) with `aws s3api` due to the fact that it can not use our optimization to skip the whole prefix. It is independant from my refactor and can be tested with the commented `s3api` tests in `test-smoke.sh`. It probably has the same weird behavior on the official AWS S3 implementation. - Considering ListMultipartUploads, I had to "abuse" upload id marker to support prefix skipping. I send an `upload-id-marker` with the hardcoded value `include` to emulate your "including" token. - Some ways to test ListMultipartUploads with existing software (my tests are limited to s3api for now). Co-authored-by: Quentin Dufour <quentin@deuxfleurs.fr> Reviewed-on: #171 Co-authored-by: Quentin <quentin@dufour.io> Co-committed-by: Quentin <quentin@dufour.io>	2022-01-12 19:04:55 +01:00
Alex Auvolat	9cb2e9e57c	Add documentation to migrate to 0.6	2022-01-10 14:42:58 +01:00
Alex Auvolat	8f39360f22	Update documentation	2022-01-07 17:14:37 +01:00
trinity-1686a	945b75dbf1	update s3 compatibility list (#177) ... Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com> Reviewed-on: #177 Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>	2021-12-15 15:05:54 +01:00
trinity-1686a	1eb972b1ac	Add compression using zstd (#173) ... fix #27 Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com> Reviewed-on: #173 Co-authored-by: trinity-1686a <trinity.pointard@gmail.com> Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>	2021-12-15 11:26:43 +01:00
Quentin Dufour	3b3a1f275f	Add a second plot	2021-12-13 11:58:03 +01:00
Quentin Dufour	dba9af2968	Update benchmark	2021-12-09 18:42:45 +01:00
Quentin Dufour	e9358054ac	Typos and dead links	2021-12-08 14:40:14 +01:00
Quentin Dufour	f9e5520ffb	Add a benchmark page with a first benchmark	2021-12-08 11:30:07 +01:00
Alex Auvolat	4b369347c0	S3 compatibility target	2021-12-06 17:18:45 +01:00
Alex Auvolat	224c89ad6e	Reorganize and improve documentation	2021-12-06 16:33:01 +01:00
Quentin Dufour	7c2037ba87	WIP front page garage	2021-12-06 15:33:43 +01:00
Alex Auvolat	c94406f428	Improve how node roles are assigned in Garage ... - change the terminology: the network configuration becomes the role table, the configuration of a nodes becomes a node's role - the modification of the role table takes place in two steps: first, changes are staged in a CRDT data structure. Then, once the user is happy with the changes, they can commit them all at once (or revert them). - update documentation - fix tests - implement smarter partition assignation algorithm This patch breaks the format of the network configuration: when migrating, the cluster will be in a state where no roles are assigned. All roles must be re-assigned and commited at once. This migration should not pose an issue.	2021-11-16 16:05:53 +01:00