diff --git a/doc/book/connect/_index.md b/doc/book/connect/_index.md index c6a46aea..b4868b9f 100644 --- a/doc/book/connect/_index.md +++ b/doc/book/connect/_index.md @@ -10,11 +10,11 @@ Garage implements the Amazon S3 protocol, which makes it compatible with many ex In particular, you will find here instructions to connect it with: - - [web applications](./apps.md) - - [website hosting](./websites.md) - - [software repositories](./repositories.md) - - [CLI tools](./cli.md) - - [your own code](./code.md) + - [web applications](@/documentation/connect/apps/index.md) + - [website hosting](@/documentation/connect/websites.md) + - [software repositories](@/documentation/connect/repositories.md) + - [CLI tools](@/documentation/connect/cli.md) + - [your own code](@/documentation/connect/code.md) ### Generic instructions @@ -31,7 +31,7 @@ you will need the following parameters: like this: `GK3515373e4c851ebaad366558` (access key), `7d37d093435a41f2aab8f13c19ba067d9776c90215f56614adad6ece597dbb34` (secret key). These keys are created and managed using the `garage` CLI, as explained in the - [quick start](../quick_start/index.md) guide. + [quick start](@/documentation/quick-start/_index.md) guide. Most S3 clients can be configured easily with these parameters, provided that you follow the following guidelines: diff --git a/doc/book/connect/cli-nextcloud-gui.png b/doc/book/connect/apps/cli-nextcloud-gui.png similarity index 100% rename from doc/book/connect/cli-nextcloud-gui.png rename to doc/book/connect/apps/cli-nextcloud-gui.png diff --git a/doc/book/connect/apps.md b/doc/book/connect/apps/index.md similarity index 98% rename from doc/book/connect/apps.md rename to doc/book/connect/apps/index.md index 65b97dfe..84f46891 100644 --- a/doc/book/connect/apps.md +++ b/doc/book/connect/apps/index.md @@ -69,7 +69,7 @@ To test your new configuration, just reload your Nextcloud webpage and start sen **From the GUI.** Activate the "External storage support" app from the "Applications" page (click on your account icon on the top right corner of your screen to display the menu). Go to your parameters page (also located below your account icon). Click on external storage (or the corresponding translation in your language). -[![Screenshot of the External Storage form](./cli-nextcloud-gui.png)](./cli-nextcloud-gui.png) +[![Screenshot of the External Storage form](cli-nextcloud-gui.png)](cli-nextcloud-gui.png) *Click on the picture to zoom* Add a new external storage. Put what you want in "folder name" (eg. "shared"). Select "Amazon S3". Keep "Access Key" for the Authentication field. @@ -256,7 +256,7 @@ Make sure you (will) have a corresponding DNS entry for them. Now we will configure a reverse proxy in front of Garage. This is required as we have no other way to serve CORS headers yet. -Check the [Configuring a reverse proxy](/cookbook/reverse_proxy.html) section to know how. +Check the [Configuring a reverse proxy](@/documentation/cookbook/reverse-proxy.md) section to know how. Now make sure that your 2 dns entries are pointing to your reverse proxy. diff --git a/doc/book/connect/fs.md b/doc/book/connect/fs.md index 60a60c7f..46df593b 100644 --- a/doc/book/connect/fs.md +++ b/doc/book/connect/fs.md @@ -14,7 +14,7 @@ Ideally, avoid these solutions at all for any serious or production use. ## rclone mount -rclone uses the same configuration when used [in CLI](/connect/cli.html) and mount mode. +rclone uses the same configuration when used [in CLI](@/documentation/connect/cli.md) and mount mode. We suppose you have the following entry in your `rclone.ini` (mine is located in `~/.config/rclone/rclone.conf`): ```toml diff --git a/doc/book/connect/websites.md b/doc/book/connect/websites.md index 1ef50463..3f62c9a6 100644 --- a/doc/book/connect/websites.md +++ b/doc/book/connect/websites.md @@ -53,7 +53,7 @@ Currently, the proposed workaround is to deploy your website manually: - Click on Get website files - You need to synchronize the output folder you see in your file explorer, we will use minio client. -Be sure that you [configured minio client](cli.html#minio-client-recommended). +Be sure that you [configured minio client](@/documentation/connect/cli.md#minio-client-recommended). Then copy this output folder @@ -66,7 +66,7 @@ mc mirror --overwrite output garage/my-site Some tools do not support sending to a S3 backend but output a compiled folder on your system. We can then use any CLI tool to upload this content to our S3 target. -First, start by [configuring minio client](cli.html#minio-client-recommended). +First, start by [configuring minio client](@/documentation/connect/cli.md#minio-client-recommended). Then build your website: diff --git a/doc/book/cookbook/_index.md b/doc/book/cookbook/_index.md index 72c32687..6e279363 100644 --- a/doc/book/cookbook/_index.md +++ b/doc/book/cookbook/_index.md @@ -9,23 +9,23 @@ A cookbook, when you cook, is a collection of recipes. Similarly, Garage's cookbook contains a collection of recipes that are known to works well! This chapter could also be referred as "Tutorials" or "Best practices". -- **[Multi-node deployment](real_world.md):** This page will walk you through all of the necessary +- **[Multi-node deployment](@/documentation/cookbook/real-world.md):** This page will walk you through all of the necessary steps to deploy Garage in a real-world setting. -- **[Building from source](from_source.md):** This page explains how to build Garage from +- **[Building from source](@/documentation/cookbook/from-source.md):** This page explains how to build Garage from source in case a binary is not provided for your architecture, or if you want to hack with us! -- **[Integration with Systemd](systemd.md):** This page explains how to run Garage +- **[Integration with Systemd](@/documentation/cookbook/systemd.md):** This page explains how to run Garage as a Systemd service (instead of as a Docker container). -- **[Configuring a gateway node](gateways.md):** This page explains how to run a gateway node in a Garage cluster, i.e. a Garage node that doesn't store data but accelerates access to data present on the other nodes. +- **[Configuring a gateway node](@/documentation/cookbook/gateways.md):** This page explains how to run a gateway node in a Garage cluster, i.e. a Garage node that doesn't store data but accelerates access to data present on the other nodes. -- **[Hosting a website](exposing_websites.md):** This page explains how to use Garage +- **[Hosting a website](@/documentation/cookbook/exposing-websites.md):** This page explains how to use Garage to host a static website. -- **[Configuring a reverse-proxy](reverse_proxy.md):** This page explains how to configure a reverse-proxy to add TLS support to your S3 api endpoint. +- **[Configuring a reverse-proxy](@/documentation/cookbook/reverse-proxy.md):** This page explains how to configure a reverse-proxy to add TLS support to your S3 api endpoint. -- **[Recovering from failures](recovering.md):** Garage's first selling point is resilience +- **[Recovering from failures](@/documentation/cookbook/recovering.md):** Garage's first selling point is resilience to hardware failures. This section explains how to recover from such a failure in the best possible way. diff --git a/doc/book/cookbook/exposing_websites.md b/doc/book/cookbook/exposing-websites.md similarity index 78% rename from doc/book/cookbook/exposing_websites.md rename to doc/book/cookbook/exposing-websites.md index dcb56d36..cc4ddfa3 100644 --- a/doc/book/cookbook/exposing_websites.md +++ b/doc/book/cookbook/exposing-websites.md @@ -14,11 +14,11 @@ Now it will be **publicly** exposed on the web endpoint (by default listening on Our website serving logic is as follow: - Supports only static websites (no support for PHP or other languages) - Does not support directory listing - - The index is defined in your `garage.toml`. ([ref](/reference_manual/configuration.html#index)) + - The index is defined in your `garage.toml`. ([ref](@/documentation/reference-manual/configuration.md#index)) Now we need to infer the URL of your website through your bucket name. Let assume: - - we set `root_domain = ".web.example.com"` in `garage.toml` ([ref](/reference_manual/configuration.html#root_domain)) + - we set `root_domain = ".web.example.com"` in `garage.toml` ([ref](@/documentation/reference-manual/configuration.md#root_domain)) - our bucket name is `garagehq.deuxfleurs.fr`. Our bucket will be served if the Host field matches one of these 2 values (the port is ignored): @@ -46,6 +46,6 @@ Now that you understand how website logic works on Garage, you can: - make the website endpoint listens on port 80 (instead of 3902) - use iptables to redirect the port 80 to the port 3902: `iptables -t nat -A PREROUTING -p tcp -dport 80 -j REDIRECT -to-port 3902` - - or configure a [reverse proxy](reverse_proxy.html) in front of Garage to add TLS (HTTPS), CORS support, etc. + - or configure a [reverse proxy](@/documentation/cookbook/reverse-proxy.md) in front of Garage to add TLS (HTTPS), CORS support, etc. -You can also take a look at [Website Integration](/connect/websites.html) to see how you can add Garage to your workflow. +You can also take a look at [Website Integration](@/documentation/connect/websites.md) to see how you can add Garage to your workflow. diff --git a/doc/book/cookbook/from_source.md b/doc/book/cookbook/from-source.md similarity index 100% rename from doc/book/cookbook/from_source.md rename to doc/book/cookbook/from-source.md diff --git a/doc/book/cookbook/real_world.md b/doc/book/cookbook/real-world.md similarity index 96% rename from doc/book/cookbook/real_world.md rename to doc/book/cookbook/real-world.md index 788c80a9..1178ded5 100644 --- a/doc/book/cookbook/real_world.md +++ b/doc/book/cookbook/real-world.md @@ -7,7 +7,7 @@ To run Garage in cluster mode, we recommend having at least 3 nodes. This will allow you to setup Garage for three-way replication of your data, the safest and most available mode proposed by Garage. -We recommend first following the [quick start guide](../quick_start/index.md) in order +We recommend first following the [quick start guide](@/documentation/quick-start/_index.md) in order to get familiar with Garage's command line and usage patterns. @@ -26,7 +26,7 @@ To run a real-world deployment, make sure the following conditions are met: to drastically reduce Garage's response times. - This guide will assume you are using Docker containers to deploy Garage on each node. - Garage can also be run independently, for instance as a [Systemd service](systemd.md). + Garage can also be run independently, for instance as a [Systemd service](@/documentation/cookbook/systemd.md). You can also use an orchestrator such as Nomad or Kubernetes to automatically manage Docker containers on a fleet of nodes. @@ -281,15 +281,15 @@ garage layout apply ``` **WARNING:** if you want to use the layout modification commands in a script, -make sure to read [this page](/reference_manual/layout.html) first. +make sure to read [this page](@/documentation/reference-manual/layout.md) first. ## Using your Garage cluster Creating buckets and managing keys is done using the `garage` CLI, -and is covered in the [quick start guide](../quick_start/index.md). +and is covered in the [quick start guide](@/documentation/quick-start/_index.md). Remember also that the CLI is self-documented thanks to the `--help` flag and the `help` subcommand (e.g. `garage help`, `garage key --help`). Configuring S3-compatible applicatiosn to interact with Garage -is covered in the [Integrations](/connect/index.html) section. +is covered in the [Integrations](@/documentation/connect/_index.md) section. diff --git a/doc/book/cookbook/reverse_proxy.md b/doc/book/cookbook/reverse-proxy.md similarity index 98% rename from doc/book/cookbook/reverse_proxy.md rename to doc/book/cookbook/reverse-proxy.md index 55a2e9b1..63ba4bbe 100644 --- a/doc/book/cookbook/reverse_proxy.md +++ b/doc/book/cookbook/reverse-proxy.md @@ -121,7 +121,7 @@ server { ### Exposing the web endpoint The web endpoint is a bit more complicated to configure as it listens on many different `Host` fields. -To better understand the logic involved, you can refer to the [Exposing buckets as websites](/cookbook/exposing_websites.html) section. +To better understand the logic involved, you can refer to the [Exposing buckets as websites](@/documentation/cookbook/exposing-websites.md) section. Also, for some applications, you may need to serve CORS headers: Garage can not serve them directly but we show how we can use nginx to serve them. You can use the following example as your starting point: diff --git a/doc/book/design/img/endpoint-latency-dc.png b/doc/book/design/benchmarks/endpoint-latency-dc.png similarity index 100% rename from doc/book/design/img/endpoint-latency-dc.png rename to doc/book/design/benchmarks/endpoint-latency-dc.png diff --git a/doc/book/design/img/endpoint-latency.png b/doc/book/design/benchmarks/endpoint-latency.png similarity index 100% rename from doc/book/design/img/endpoint-latency.png rename to doc/book/design/benchmarks/endpoint-latency.png diff --git a/doc/book/design/benchmarks.md b/doc/book/design/benchmarks/index.md similarity index 97% rename from doc/book/design/benchmarks.md rename to doc/book/design/benchmarks/index.md index c47995b4..c2215a4a 100644 --- a/doc/book/design/benchmarks.md +++ b/doc/book/design/benchmarks/index.md @@ -28,7 +28,7 @@ We selected 5 standard endpoints that are often in the critical path: ListBucket In this first benchmark, we consider 5 instances that are located in a different place each. To simulate the distance, we configure mknet with a RTT between each node of 100 ms +/- 20 ms of jitter. We get the following graph, where the colored bars represent the mean latency while the error bars the minimum and maximum one: -![Comparison of endpoints latency for minio and garage](./img/endpoint-latency.png) +![Comparison of endpoints latency for minio and garage](./endpoint-latency.png) Compared to garage, minio latency drastically increases on 3 endpoints: GetObject, PutObject, RemoveObject. @@ -46,7 +46,7 @@ We consider that intra-DC communications are now very cheap with a latency of 0. The inter-DC remains costly with the same value as before (100ms +/- 20ms of jitter). We plot a similar graph as before: -![Comparison of endpoints latency for minio and garage with 6 nodes in 3 DC](./img/endpoint-latency-dc.png) +![Comparison of endpoints latency for minio and garage with 6 nodes in 3 DC](./endpoint-latency-dc.png) This new graph is very similar to the one before, neither minio or garage seems to benefit from this new topology, but they also do not suffer from it. diff --git a/doc/book/design/internals.md b/doc/book/design/internals.md index be531e97..05d852e2 100644 --- a/doc/book/design/internals.md +++ b/doc/book/design/internals.md @@ -17,7 +17,7 @@ In the meantime, you can find some information at the following links: - [this presentation (in French)](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/doc/talks/2020-12-02_wide-team/talk.pdf) -- [an old design draft](/working_documents/design_draft.md) +- [an old design draft](@/documentation/working-documents/design-draft.md) ## Garbage collection diff --git a/doc/book/design/related_work.md b/doc/book/design/related-work.md similarity index 100% rename from doc/book/design/related_work.md rename to doc/book/design/related-work.md diff --git a/doc/book/development/miscellaneous_notes.md b/doc/book/development/miscellaneous-notes.md similarity index 100% rename from doc/book/development/miscellaneous_notes.md rename to doc/book/development/miscellaneous-notes.md diff --git a/doc/book/development/release_process.md b/doc/book/development/release-process.md similarity index 100% rename from doc/book/development/release_process.md rename to doc/book/development/release-process.md diff --git a/doc/book/development/scripts.md b/doc/book/development/scripts.md index 25f2ce0e..498419e2 100644 --- a/doc/book/development/scripts.md +++ b/doc/book/development/scripts.md @@ -34,7 +34,7 @@ You can inspect the detailed configuration, including ports, by inspecting `/tmp This script also spawns a simple HTTPS reverse proxy through `socat` for the S3 endpoint that listens on port `4443`. Some libraries might require a TLS endpoint to work, refer to our issue [#64](https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/64) for more detailed information on this subject. -This script covers the [Launching the garage server](/quick_start/index.html#launching-the-garage-server) section of our Quick start page. +This script covers the [Launching the garage server](@/documentation/quick-start/_index.md#launching-the-garage-server) section of our Quick start page. ### 2. Make them join the cluster @@ -44,7 +44,7 @@ This script covers the [Launching the garage server](/quick_start/index.html#lau This script will configure each instance by assigning them a zone (`dc1`) and a weight (`1`). -This script covers the [Configuring your Garage node](/quick_start/index.html#configuring-your-garage-node) section of our Quick start page. +This script covers the [Creating a cluster layout](@/documentation/quick-start/_index.md#creating-a-cluster-layout) section of our Quick start page. ### 3. Create a key and a bucket @@ -55,7 +55,7 @@ This script covers the [Configuring your Garage node](/quick_start/index.html#co This script will create a bucket named `eprouvette` with a key having read and write rights on this bucket. The key is stored in a filed named `/tmp/garage.s3` and can be used by the following tools to pre-configure them. -This script covers the [Creating buckets and keys](/quick_start/index.html#creating-buckets-and-keys) section of our Quick start page. +This script covers the [Creating buckets and keys](@/documentation/quick-start/_index.md#creating-buckets-and-keys) section of our Quick start page. ## Handlers for generic tools diff --git a/doc/book/quick-start/_index.md b/doc/book/quick-start/_index.md index 9c3f7315..53ef00b4 100644 --- a/doc/book/quick-start/_index.md +++ b/doc/book/quick-start/_index.md @@ -11,7 +11,7 @@ and how to interact with it. Our goal is to introduce you to Garage's workflows. Following this guide is recommended before moving on to -[configuring a multi-node cluster](../cookbook/real_world.md). +[configuring a multi-node cluster](@/documentation/cookbook/real-world.md). Note that this kind of deployment should not be used in production, as it provides no redundancy for your data! @@ -28,10 +28,12 @@ or in `~/.local/bin`). If a binary of the last version is not available for your architecture, or if you want a build customized for your system, -you can [build Garage from source](../cookbook/from_source.md). +you can [build Garage from source](@/documentation/cookbook/from-source.md). -## Writing a first configuration file +## Configuring and starting Garage + +### Writing a first configuration file This first configuration file should allow you to get started easily with the simplest possible Garage deployment. @@ -73,12 +75,12 @@ Garage server will not be persistent. Change these to locations on your local di your data to be persisted properly. -## Launching the Garage server +### Launching the Garage server Use the following command to launch the Garage server with our configuration file: ``` -RUST_LOG=garage=info garage server +garage server ``` You can tune Garage's verbosity as follows (from less verbose to more verbose): @@ -89,11 +91,11 @@ RUST_LOG=garage=debug garage server RUST_LOG=garage=trace garage server ``` -Log level `info` is recommended for most use cases. +Log level `info` is the default value and is recommended for most use cases. Log level `debug` can help you check why your S3 API calls are not working. -## Checking that Garage runs correctly +### Checking that Garage runs correctly The `garage` utility is also used as a CLI tool to configure your Garage deployment. It uses values from the TOML configuration file to find the Garage daemon running on the @@ -152,7 +154,7 @@ garage help garage bucket allow --help ``` -#### Create a bucket +### Create a bucket Let's take an example where we want to deploy NextCloud using Garage as the main data storage. @@ -170,7 +172,7 @@ garage bucket list garage bucket info nextcloud-bucket ``` -#### Create an API key +### Create an API key The `nextcloud-bucket` bucket now exists on the Garage server, however it cannot be accessed until we add an API key with the proper access rights. @@ -200,7 +202,7 @@ garage key list garage key info nextcloud-app-key ``` -#### Allow a key to access a bucket +### Allow a key to access a bucket Now that we have a bucket and a key, we need to give permissions to the key on the bucket: @@ -229,7 +231,7 @@ Before reading the following, you need a working `mc` command on your path. Note that on certain Linux distributions such as Arch Linux, the Minio client binary is called `mcli` instead of `mc` (to avoid name clashes with the Midnight Commander). -#### Configure `mc` +### Configure `mc` You need your access key and secret key created above. We will assume you are invoking `mc` on the same machine as the Garage server, @@ -257,7 +259,7 @@ or `$HOME/.bashrc` file: export MC_REGION=garage ``` -#### Use `mc` +### Use `mc` You can not list buckets from `mc` currently. @@ -271,7 +273,7 @@ mc mirror localdir/ my-garage/another-bucket ``` -#### Other tools for interacting with Garage +### Other tools for interacting with Garage The following tools can also be used to send and recieve files from/to Garage: @@ -280,5 +282,5 @@ The following tools can also be used to send and recieve files from/to Garage: - [Cyberduck](https://cyberduck.io/) - [`s3cmd`](https://s3tools.org/s3cmd) -Refer to the ["Integrations" section](../connect/index.md) to learn how to +Refer to the ["Integrations" section](@/documentation/connect/_index.md) to learn how to configure application and command line utilities to integrate with Garage. diff --git a/doc/book/reference-manual/configuration.md b/doc/book/reference-manual/configuration.md index b9c093aa..3f553055 100644 --- a/doc/book/reference-manual/configuration.md +++ b/doc/book/reference-manual/configuration.md @@ -60,7 +60,7 @@ Store this folder on a fast SSD drive if possible to maximize Garage's performan The directory in which Garage will store the data blocks of objects. This folder can be placed on an HDD. The space available for `data_dir` should be counted to determine a node's capacity -when [configuring it](../getting_started/05_cluster.md). +when [configuring it](@/documentation/cookbook/real-world.md). #### `block_size` @@ -206,7 +206,7 @@ Garage will accept S3 API calls that are targetted to the S3 region defined here API calls targetted to other regions will fail with a AuthorizationHeaderMalformed error message that redirects the client to the correct region. -#### `root_domain` +#### `root_domain` {#root_domain} The optionnal suffix to access bucket using vhost-style in addition to path-style request. Note path-style requests are always enabled, whether or not vhost-style is configured. diff --git a/doc/book/reference-manual/layout.md b/doc/book/reference-manual/layout.md index 3f818b62..7debbf33 100644 --- a/doc/book/reference-manual/layout.md +++ b/doc/book/reference-manual/layout.md @@ -7,7 +7,7 @@ The cluster layout in Garage is a table that assigns to each node a role in the cluster. The role of a node in Garage can either be a storage node with a certain capacity, or a gateway node that does not store data and is only used as an API entry point for faster cluster access. -An introduction to building cluster layouts can be found in the [production deployment](/cookbook/real_world.md) page. +An introduction to building cluster layouts can be found in the [production deployment](@/documentation/cookbook/real-world.md) page. ## How cluster layouts work in Garage diff --git a/doc/book/reference-manual/s3_compatibility.md b/doc/book/reference-manual/s3-compatibility.md similarity index 100% rename from doc/book/reference-manual/s3_compatibility.md rename to doc/book/reference-manual/s3-compatibility.md diff --git a/doc/book/working-documents/compatibility_target.md b/doc/book/working-documents/compatibility-target.md similarity index 100% rename from doc/book/working-documents/compatibility_target.md rename to doc/book/working-documents/compatibility-target.md diff --git a/doc/book/working-documents/design_draft.md b/doc/book/working-documents/design-draft.md similarity index 100% rename from doc/book/working-documents/design_draft.md rename to doc/book/working-documents/design-draft.md diff --git a/doc/book/working-documents/load_balancing.md b/doc/book/working-documents/load-balancing.md similarity index 100% rename from doc/book/working-documents/load_balancing.md rename to doc/book/working-documents/load-balancing.md diff --git a/doc/book/working-documents/migration_04.md b/doc/book/working-documents/migration-04.md similarity index 100% rename from doc/book/working-documents/migration_04.md rename to doc/book/working-documents/migration-04.md diff --git a/doc/book/working-documents/migration_06.md b/doc/book/working-documents/migration-06.md similarity index 100% rename from doc/book/working-documents/migration_06.md rename to doc/book/working-documents/migration-06.md