reorganize documentation for new website #213

Merged
lx merged 7 commits from doc-new-website into main 2022-02-07 10:51:12 +00:00
27 changed files with 53 additions and 51 deletions
Showing only changes of commit 961a4cf7b7 - Show all commits

View file

@ -10,11 +10,11 @@ Garage implements the Amazon S3 protocol, which makes it compatible with many ex
In particular, you will find here instructions to connect it with: In particular, you will find here instructions to connect it with:
- [web applications](./apps.md) - [web applications](@/documentation/connect/apps/index.md)
- [website hosting](./websites.md) - [website hosting](@/documentation/connect/websites.md)
- [software repositories](./repositories.md) - [software repositories](@/documentation/connect/repositories.md)
- [CLI tools](./cli.md) - [CLI tools](@/documentation/connect/cli.md)
- [your own code](./code.md) - [your own code](@/documentation/connect/code.md)
### Generic instructions ### Generic instructions
@ -31,7 +31,7 @@ you will need the following parameters:
like this: `GK3515373e4c851ebaad366558` (access key), like this: `GK3515373e4c851ebaad366558` (access key),
`7d37d093435a41f2aab8f13c19ba067d9776c90215f56614adad6ece597dbb34` (secret key). `7d37d093435a41f2aab8f13c19ba067d9776c90215f56614adad6ece597dbb34` (secret key).
These keys are created and managed using the `garage` CLI, as explained in the These keys are created and managed using the `garage` CLI, as explained in the
[quick start](../quick_start/index.md) guide. [quick start](@/documentation/quick-start/_index.md) guide.
Most S3 clients can be configured easily with these parameters, Most S3 clients can be configured easily with these parameters,
provided that you follow the following guidelines: provided that you follow the following guidelines:

View file

@ -69,7 +69,7 @@ To test your new configuration, just reload your Nextcloud webpage and start sen
**From the GUI.** Activate the "External storage support" app from the "Applications" page (click on your account icon on the top right corner of your screen to display the menu). Go to your parameters page (also located below your account icon). Click on external storage (or the corresponding translation in your language). **From the GUI.** Activate the "External storage support" app from the "Applications" page (click on your account icon on the top right corner of your screen to display the menu). Go to your parameters page (also located below your account icon). Click on external storage (or the corresponding translation in your language).
[![Screenshot of the External Storage form](./cli-nextcloud-gui.png)](./cli-nextcloud-gui.png) [![Screenshot of the External Storage form](cli-nextcloud-gui.png)](cli-nextcloud-gui.png)
*Click on the picture to zoom* *Click on the picture to zoom*
Add a new external storage. Put what you want in "folder name" (eg. "shared"). Select "Amazon S3". Keep "Access Key" for the Authentication field. Add a new external storage. Put what you want in "folder name" (eg. "shared"). Select "Amazon S3". Keep "Access Key" for the Authentication field.
@ -256,7 +256,7 @@ Make sure you (will) have a corresponding DNS entry for them.
Now we will configure a reverse proxy in front of Garage. Now we will configure a reverse proxy in front of Garage.
This is required as we have no other way to serve CORS headers yet. This is required as we have no other way to serve CORS headers yet.
Check the [Configuring a reverse proxy](/cookbook/reverse_proxy.html) section to know how. Check the [Configuring a reverse proxy](@/documentation/cookbook/reverse-proxy.md) section to know how.
Now make sure that your 2 dns entries are pointing to your reverse proxy. Now make sure that your 2 dns entries are pointing to your reverse proxy.

View file

@ -14,7 +14,7 @@ Ideally, avoid these solutions at all for any serious or production use.
## rclone mount ## rclone mount
rclone uses the same configuration when used [in CLI](/connect/cli.html) and mount mode. rclone uses the same configuration when used [in CLI](@/documentation/connect/cli.md) and mount mode.
We suppose you have the following entry in your `rclone.ini` (mine is located in `~/.config/rclone/rclone.conf`): We suppose you have the following entry in your `rclone.ini` (mine is located in `~/.config/rclone/rclone.conf`):
```toml ```toml

View file

@ -53,7 +53,7 @@ Currently, the proposed workaround is to deploy your website manually:
- Click on Get website files - Click on Get website files
- You need to synchronize the output folder you see in your file explorer, we will use minio client. - You need to synchronize the output folder you see in your file explorer, we will use minio client.
Be sure that you [configured minio client](cli.html#minio-client-recommended). Be sure that you [configured minio client](@/documentation/connect/cli.md#minio-client-recommended).
Then copy this output folder Then copy this output folder
@ -66,7 +66,7 @@ mc mirror --overwrite output garage/my-site
Some tools do not support sending to a S3 backend but output a compiled folder on your system. Some tools do not support sending to a S3 backend but output a compiled folder on your system.
We can then use any CLI tool to upload this content to our S3 target. We can then use any CLI tool to upload this content to our S3 target.
First, start by [configuring minio client](cli.html#minio-client-recommended). First, start by [configuring minio client](@/documentation/connect/cli.md#minio-client-recommended).
Then build your website: Then build your website:

View file

@ -9,23 +9,23 @@ A cookbook, when you cook, is a collection of recipes.
Similarly, Garage's cookbook contains a collection of recipes that are known to works well! Similarly, Garage's cookbook contains a collection of recipes that are known to works well!
This chapter could also be referred as "Tutorials" or "Best practices". This chapter could also be referred as "Tutorials" or "Best practices".
- **[Multi-node deployment](real_world.md):** This page will walk you through all of the necessary - **[Multi-node deployment](@/documentation/cookbook/real-world.md):** This page will walk you through all of the necessary
steps to deploy Garage in a real-world setting. steps to deploy Garage in a real-world setting.
- **[Building from source](from_source.md):** This page explains how to build Garage from - **[Building from source](@/documentation/cookbook/from-source.md):** This page explains how to build Garage from
source in case a binary is not provided for your architecture, or if you want to source in case a binary is not provided for your architecture, or if you want to
hack with us! hack with us!
- **[Integration with Systemd](systemd.md):** This page explains how to run Garage - **[Integration with Systemd](@/documentation/cookbook/systemd.md):** This page explains how to run Garage
as a Systemd service (instead of as a Docker container). as a Systemd service (instead of as a Docker container).
- **[Configuring a gateway node](gateways.md):** This page explains how to run a gateway node in a Garage cluster, i.e. a Garage node that doesn't store data but accelerates access to data present on the other nodes. - **[Configuring a gateway node](@/documentation/cookbook/gateways.md):** This page explains how to run a gateway node in a Garage cluster, i.e. a Garage node that doesn't store data but accelerates access to data present on the other nodes.
- **[Hosting a website](exposing_websites.md):** This page explains how to use Garage - **[Hosting a website](@/documentation/cookbook/exposing-websites.md):** This page explains how to use Garage
to host a static website. to host a static website.
- **[Configuring a reverse-proxy](reverse_proxy.md):** This page explains how to configure a reverse-proxy to add TLS support to your S3 api endpoint. - **[Configuring a reverse-proxy](@/documentation/cookbook/reverse-proxy.md):** This page explains how to configure a reverse-proxy to add TLS support to your S3 api endpoint.
- **[Recovering from failures](recovering.md):** Garage's first selling point is resilience - **[Recovering from failures](@/documentation/cookbook/recovering.md):** Garage's first selling point is resilience
to hardware failures. This section explains how to recover from such a failure in the to hardware failures. This section explains how to recover from such a failure in the
best possible way. best possible way.

View file

@ -14,11 +14,11 @@ Now it will be **publicly** exposed on the web endpoint (by default listening on
Our website serving logic is as follow: Our website serving logic is as follow:
- Supports only static websites (no support for PHP or other languages) - Supports only static websites (no support for PHP or other languages)
- Does not support directory listing - Does not support directory listing
- The index is defined in your `garage.toml`. ([ref](/reference_manual/configuration.html#index)) - The index is defined in your `garage.toml`. ([ref](@/documentation/reference-manual/configuration.md#index))
Now we need to infer the URL of your website through your bucket name. Now we need to infer the URL of your website through your bucket name.
Let assume: Let assume:
- we set `root_domain = ".web.example.com"` in `garage.toml` ([ref](/reference_manual/configuration.html#root_domain)) - we set `root_domain = ".web.example.com"` in `garage.toml` ([ref](@/documentation/reference-manual/configuration.md#root_domain))
- our bucket name is `garagehq.deuxfleurs.fr`. - our bucket name is `garagehq.deuxfleurs.fr`.
Our bucket will be served if the Host field matches one of these 2 values (the port is ignored): Our bucket will be served if the Host field matches one of these 2 values (the port is ignored):
@ -46,6 +46,6 @@ Now that you understand how website logic works on Garage, you can:
- make the website endpoint listens on port 80 (instead of 3902) - make the website endpoint listens on port 80 (instead of 3902)
- use iptables to redirect the port 80 to the port 3902: - use iptables to redirect the port 80 to the port 3902:
`iptables -t nat -A PREROUTING -p tcp -dport 80 -j REDIRECT -to-port 3902` `iptables -t nat -A PREROUTING -p tcp -dport 80 -j REDIRECT -to-port 3902`
- or configure a [reverse proxy](reverse_proxy.html) in front of Garage to add TLS (HTTPS), CORS support, etc. - or configure a [reverse proxy](@/documentation/cookbook/reverse-proxy.md) in front of Garage to add TLS (HTTPS), CORS support, etc.
You can also take a look at [Website Integration](/connect/websites.html) to see how you can add Garage to your workflow. You can also take a look at [Website Integration](@/documentation/connect/websites.md) to see how you can add Garage to your workflow.

View file

@ -7,7 +7,7 @@ To run Garage in cluster mode, we recommend having at least 3 nodes.
This will allow you to setup Garage for three-way replication of your data, This will allow you to setup Garage for three-way replication of your data,
the safest and most available mode proposed by Garage. the safest and most available mode proposed by Garage.
We recommend first following the [quick start guide](../quick_start/index.md) in order We recommend first following the [quick start guide](@/documentation/quick-start/_index.md) in order
to get familiar with Garage's command line and usage patterns. to get familiar with Garage's command line and usage patterns.
@ -26,7 +26,7 @@ To run a real-world deployment, make sure the following conditions are met:
to drastically reduce Garage's response times. to drastically reduce Garage's response times.
- This guide will assume you are using Docker containers to deploy Garage on each node. - This guide will assume you are using Docker containers to deploy Garage on each node.
Garage can also be run independently, for instance as a [Systemd service](systemd.md). Garage can also be run independently, for instance as a [Systemd service](@/documentation/cookbook/systemd.md).
You can also use an orchestrator such as Nomad or Kubernetes to automatically manage You can also use an orchestrator such as Nomad or Kubernetes to automatically manage
Docker containers on a fleet of nodes. Docker containers on a fleet of nodes.
@ -281,15 +281,15 @@ garage layout apply
``` ```
**WARNING:** if you want to use the layout modification commands in a script, **WARNING:** if you want to use the layout modification commands in a script,
make sure to read [this page](/reference_manual/layout.html) first. make sure to read [this page](@/documentation/reference-manual/layout.md) first.
## Using your Garage cluster ## Using your Garage cluster
Creating buckets and managing keys is done using the `garage` CLI, Creating buckets and managing keys is done using the `garage` CLI,
and is covered in the [quick start guide](../quick_start/index.md). and is covered in the [quick start guide](@/documentation/quick-start/_index.md).
Remember also that the CLI is self-documented thanks to the `--help` flag and Remember also that the CLI is self-documented thanks to the `--help` flag and
the `help` subcommand (e.g. `garage help`, `garage key --help`). the `help` subcommand (e.g. `garage help`, `garage key --help`).
Configuring S3-compatible applicatiosn to interact with Garage Configuring S3-compatible applicatiosn to interact with Garage
is covered in the [Integrations](/connect/index.html) section. is covered in the [Integrations](@/documentation/connect/_index.md) section.

View file

@ -121,7 +121,7 @@ server {
### Exposing the web endpoint ### Exposing the web endpoint
The web endpoint is a bit more complicated to configure as it listens on many different `Host` fields. The web endpoint is a bit more complicated to configure as it listens on many different `Host` fields.
To better understand the logic involved, you can refer to the [Exposing buckets as websites](/cookbook/exposing_websites.html) section. To better understand the logic involved, you can refer to the [Exposing buckets as websites](@/documentation/cookbook/exposing-websites.md) section.
Also, for some applications, you may need to serve CORS headers: Garage can not serve them directly but we show how we can use nginx to serve them. Also, for some applications, you may need to serve CORS headers: Garage can not serve them directly but we show how we can use nginx to serve them.
You can use the following example as your starting point: You can use the following example as your starting point:

View file

@ -28,7 +28,7 @@ We selected 5 standard endpoints that are often in the critical path: ListBucket
In this first benchmark, we consider 5 instances that are located in a different place each. To simulate the distance, we configure mknet with a RTT between each node of 100 ms +/- 20 ms of jitter. We get the following graph, where the colored bars represent the mean latency while the error bars the minimum and maximum one: In this first benchmark, we consider 5 instances that are located in a different place each. To simulate the distance, we configure mknet with a RTT between each node of 100 ms +/- 20 ms of jitter. We get the following graph, where the colored bars represent the mean latency while the error bars the minimum and maximum one:
![Comparison of endpoints latency for minio and garage](./img/endpoint-latency.png) ![Comparison of endpoints latency for minio and garage](./endpoint-latency.png)
Compared to garage, minio latency drastically increases on 3 endpoints: GetObject, PutObject, RemoveObject. Compared to garage, minio latency drastically increases on 3 endpoints: GetObject, PutObject, RemoveObject.
@ -46,7 +46,7 @@ We consider that intra-DC communications are now very cheap with a latency of 0.
The inter-DC remains costly with the same value as before (100ms +/- 20ms of jitter). The inter-DC remains costly with the same value as before (100ms +/- 20ms of jitter).
We plot a similar graph as before: We plot a similar graph as before:
![Comparison of endpoints latency for minio and garage with 6 nodes in 3 DC](./img/endpoint-latency-dc.png) ![Comparison of endpoints latency for minio and garage with 6 nodes in 3 DC](./endpoint-latency-dc.png)
This new graph is very similar to the one before, neither minio or garage seems to benefit from this new topology, but they also do not suffer from it. This new graph is very similar to the one before, neither minio or garage seems to benefit from this new topology, but they also do not suffer from it.

View file

@ -17,7 +17,7 @@ In the meantime, you can find some information at the following links:
- [this presentation (in French)](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/doc/talks/2020-12-02_wide-team/talk.pdf) - [this presentation (in French)](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/doc/talks/2020-12-02_wide-team/talk.pdf)
- [an old design draft](/working_documents/design_draft.md) - [an old design draft](@/documentation/working-documents/design-draft.md)
## Garbage collection ## Garbage collection

View file

@ -34,7 +34,7 @@ You can inspect the detailed configuration, including ports, by inspecting `/tmp
This script also spawns a simple HTTPS reverse proxy through `socat` for the S3 endpoint that listens on port `4443`. This script also spawns a simple HTTPS reverse proxy through `socat` for the S3 endpoint that listens on port `4443`.
Some libraries might require a TLS endpoint to work, refer to our issue [#64](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/64) for more detailed information on this subject. Some libraries might require a TLS endpoint to work, refer to our issue [#64](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/64) for more detailed information on this subject.
This script covers the [Launching the garage server](/quick_start/index.html#launching-the-garage-server) section of our Quick start page. This script covers the [Launching the garage server](@/documentation/quick-start/_index.md#launching-the-garage-server) section of our Quick start page.
### 2. Make them join the cluster ### 2. Make them join the cluster
@ -44,7 +44,7 @@ This script covers the [Launching the garage server](/quick_start/index.html#lau
This script will configure each instance by assigning them a zone (`dc1`) and a weight (`1`). This script will configure each instance by assigning them a zone (`dc1`) and a weight (`1`).
This script covers the [Configuring your Garage node](/quick_start/index.html#configuring-your-garage-node) section of our Quick start page. This script covers the [Creating a cluster layout](@/documentation/quick-start/_index.md#creating-a-cluster-layout) section of our Quick start page.
### 3. Create a key and a bucket ### 3. Create a key and a bucket
@ -55,7 +55,7 @@ This script covers the [Configuring your Garage node](/quick_start/index.html#co
This script will create a bucket named `eprouvette` with a key having read and write rights on this bucket. This script will create a bucket named `eprouvette` with a key having read and write rights on this bucket.
The key is stored in a filed named `/tmp/garage.s3` and can be used by the following tools to pre-configure them. The key is stored in a filed named `/tmp/garage.s3` and can be used by the following tools to pre-configure them.
This script covers the [Creating buckets and keys](/quick_start/index.html#creating-buckets-and-keys) section of our Quick start page. This script covers the [Creating buckets and keys](@/documentation/quick-start/_index.md#creating-buckets-and-keys) section of our Quick start page.
## Handlers for generic tools ## Handlers for generic tools

View file

@ -11,7 +11,7 @@ and how to interact with it.
Our goal is to introduce you to Garage's workflows. Our goal is to introduce you to Garage's workflows.
Following this guide is recommended before moving on to Following this guide is recommended before moving on to
[configuring a multi-node cluster](../cookbook/real_world.md). [configuring a multi-node cluster](@/documentation/cookbook/real-world.md).
Note that this kind of deployment should not be used in production, Note that this kind of deployment should not be used in production,
as it provides no redundancy for your data! as it provides no redundancy for your data!
@ -28,10 +28,12 @@ or in `~/.local/bin`).
If a binary of the last version is not available for your architecture, If a binary of the last version is not available for your architecture,
or if you want a build customized for your system, or if you want a build customized for your system,
you can [build Garage from source](../cookbook/from_source.md). you can [build Garage from source](@/documentation/cookbook/from-source.md).
## Writing a first configuration file ## Configuring and starting Garage
### Writing a first configuration file
This first configuration file should allow you to get started easily with the simplest This first configuration file should allow you to get started easily with the simplest
possible Garage deployment. possible Garage deployment.
@ -73,12 +75,12 @@ Garage server will not be persistent. Change these to locations on your local di
your data to be persisted properly. your data to be persisted properly.
## Launching the Garage server ### Launching the Garage server
Use the following command to launch the Garage server with our configuration file: Use the following command to launch the Garage server with our configuration file:
``` ```
RUST_LOG=garage=info garage server garage server
``` ```
You can tune Garage's verbosity as follows (from less verbose to more verbose): You can tune Garage's verbosity as follows (from less verbose to more verbose):
@ -89,11 +91,11 @@ RUST_LOG=garage=debug garage server
RUST_LOG=garage=trace garage server RUST_LOG=garage=trace garage server
``` ```
Log level `info` is recommended for most use cases. Log level `info` is the default value and is recommended for most use cases.
Log level `debug` can help you check why your S3 API calls are not working. Log level `debug` can help you check why your S3 API calls are not working.
## Checking that Garage runs correctly ### Checking that Garage runs correctly
The `garage` utility is also used as a CLI tool to configure your Garage deployment. The `garage` utility is also used as a CLI tool to configure your Garage deployment.
It uses values from the TOML configuration file to find the Garage daemon running on the It uses values from the TOML configuration file to find the Garage daemon running on the
@ -152,7 +154,7 @@ garage help
garage bucket allow --help garage bucket allow --help
``` ```
#### Create a bucket ### Create a bucket
Let's take an example where we want to deploy NextCloud using Garage as the Let's take an example where we want to deploy NextCloud using Garage as the
main data storage. main data storage.
@ -170,7 +172,7 @@ garage bucket list
garage bucket info nextcloud-bucket garage bucket info nextcloud-bucket
``` ```
#### Create an API key ### Create an API key
The `nextcloud-bucket` bucket now exists on the Garage server, The `nextcloud-bucket` bucket now exists on the Garage server,
however it cannot be accessed until we add an API key with the proper access rights. however it cannot be accessed until we add an API key with the proper access rights.
@ -200,7 +202,7 @@ garage key list
garage key info nextcloud-app-key garage key info nextcloud-app-key
``` ```
#### Allow a key to access a bucket ### Allow a key to access a bucket
Now that we have a bucket and a key, we need to give permissions to the key on the bucket: Now that we have a bucket and a key, we need to give permissions to the key on the bucket:
@ -229,7 +231,7 @@ Before reading the following, you need a working `mc` command on your path.
Note that on certain Linux distributions such as Arch Linux, the Minio client binary Note that on certain Linux distributions such as Arch Linux, the Minio client binary
is called `mcli` instead of `mc` (to avoid name clashes with the Midnight Commander). is called `mcli` instead of `mc` (to avoid name clashes with the Midnight Commander).
#### Configure `mc` ### Configure `mc`
You need your access key and secret key created above. You need your access key and secret key created above.
We will assume you are invoking `mc` on the same machine as the Garage server, We will assume you are invoking `mc` on the same machine as the Garage server,
@ -257,7 +259,7 @@ or `$HOME/.bashrc` file:
export MC_REGION=garage export MC_REGION=garage
``` ```
#### Use `mc` ### Use `mc`
You can not list buckets from `mc` currently. You can not list buckets from `mc` currently.
@ -271,7 +273,7 @@ mc mirror localdir/ my-garage/another-bucket
``` ```
#### Other tools for interacting with Garage ### Other tools for interacting with Garage
The following tools can also be used to send and recieve files from/to Garage: The following tools can also be used to send and recieve files from/to Garage:
@ -280,5 +282,5 @@ The following tools can also be used to send and recieve files from/to Garage:
- [Cyberduck](https://cyberduck.io/) - [Cyberduck](https://cyberduck.io/)
- [`s3cmd`](https://s3tools.org/s3cmd) - [`s3cmd`](https://s3tools.org/s3cmd)
Refer to the ["Integrations" section](../connect/index.md) to learn how to Refer to the ["Integrations" section](@/documentation/connect/_index.md) to learn how to
configure application and command line utilities to integrate with Garage. configure application and command line utilities to integrate with Garage.

View file

@ -60,7 +60,7 @@ Store this folder on a fast SSD drive if possible to maximize Garage's performan
The directory in which Garage will store the data blocks of objects. The directory in which Garage will store the data blocks of objects.
This folder can be placed on an HDD. The space available for `data_dir` This folder can be placed on an HDD. The space available for `data_dir`
should be counted to determine a node's capacity should be counted to determine a node's capacity
when [configuring it](../getting_started/05_cluster.md). when [configuring it](@/documentation/cookbook/real-world.md).
#### `block_size` #### `block_size`
@ -206,7 +206,7 @@ Garage will accept S3 API calls that are targetted to the S3 region defined here
API calls targetted to other regions will fail with a AuthorizationHeaderMalformed error API calls targetted to other regions will fail with a AuthorizationHeaderMalformed error
message that redirects the client to the correct region. message that redirects the client to the correct region.
#### `root_domain` #### `root_domain` {#root_domain}
The optionnal suffix to access bucket using vhost-style in addition to path-style request. The optionnal suffix to access bucket using vhost-style in addition to path-style request.
Note path-style requests are always enabled, whether or not vhost-style is configured. Note path-style requests are always enabled, whether or not vhost-style is configured.

View file

@ -7,7 +7,7 @@ The cluster layout in Garage is a table that assigns to each node a role in
the cluster. The role of a node in Garage can either be a storage node with the cluster. The role of a node in Garage can either be a storage node with
a certain capacity, or a gateway node that does not store data and is only a certain capacity, or a gateway node that does not store data and is only
used as an API entry point for faster cluster access. used as an API entry point for faster cluster access.
An introduction to building cluster layouts can be found in the [production deployment](/cookbook/real_world.md) page. An introduction to building cluster layouts can be found in the [production deployment](@/documentation/cookbook/real-world.md) page.
## How cluster layouts work in Garage ## How cluster layouts work in Garage