forked from Deuxfleurs/garage
Finish writing about Garage features, and fix from-source instructions
This commit is contained in:
parent
f6aebefcc9
commit
1d0a610690
3 changed files with 110 additions and 49 deletions
|
@ -5,12 +5,14 @@ weight = 25
|
||||||
|
|
||||||
## Configuring a bucket for website access
|
## Configuring a bucket for website access
|
||||||
|
|
||||||
There are two methods to expose buckets as website:
|
There are three methods to expose buckets as website:
|
||||||
|
|
||||||
1. using the PutBucketWebsite S3 API call, which is allowed for access keys that have the owner permission bit set
|
1. using the PutBucketWebsite S3 API call, which is allowed for access keys that have the owner permission bit set
|
||||||
|
|
||||||
2. from the Garage CLI, by an adminstrator of the cluster
|
2. from the Garage CLI, by an adminstrator of the cluster
|
||||||
|
|
||||||
|
3. using the Garage administration API
|
||||||
|
|
||||||
The `PutBucketWebsite` API endpoint [is documented](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketWebsite.html) in the official AWS docs.
|
The `PutBucketWebsite` API endpoint [is documented](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketWebsite.html) in the official AWS docs.
|
||||||
This endpoint can also be called [using `aws s3api`](https://docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-website.html) on the command line.
|
This endpoint can also be called [using `aws s3api`](https://docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-website.html) on the command line.
|
||||||
The website configuration supported by Garage is only a subset of the possibilities on Amazon S3: redirections are not supported, only the index document and error document can be specified.
|
The website configuration supported by Garage is only a subset of the possibilities on Amazon S3: redirections are not supported, only the index document and error document can be specified.
|
||||||
|
|
|
@ -20,57 +20,76 @@ sudo apt-get update
|
||||||
sudo apt-get install build-essential
|
sudo apt-get install build-essential
|
||||||
```
|
```
|
||||||
|
|
||||||
## Using source from the Gitea repository (recommended)
|
## Building from source from the Gitea repository
|
||||||
|
|
||||||
The primary location for Garage's source code is the
|
The primary location for Garage's source code is the
|
||||||
[Gitea repository](https://git.deuxfleurs.fr/Deuxfleurs/garage).
|
[Gitea repository](https://git.deuxfleurs.fr/Deuxfleurs/garage),
|
||||||
|
which contains all of the released versions as well as the code
|
||||||
|
for the developpement of the next version.
|
||||||
|
|
||||||
Clone the repository and build Garage with the following commands:
|
Clone the repository and enter it as follows:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://git.deuxfleurs.fr/Deuxfleurs/garage.git
|
git clone https://git.deuxfleurs.fr/Deuxfleurs/garage.git
|
||||||
cd garage
|
cd garage
|
||||||
cargo build
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Be careful, as this will make a debug build of Garage, which will be extremely slow!
|
If you wish to build a specific version of Garage, check out the corresponding tag. For instance:
|
||||||
To make a release build, invoke `cargo build --release` (this takes much longer).
|
|
||||||
|
|
||||||
The binaries built this way are found in `target/{debug,release}/garage`.
|
|
||||||
|
|
||||||
## Using source from `crates.io`
|
|
||||||
|
|
||||||
Garage's source code is published on `crates.io`, Rust's official package repository.
|
|
||||||
This means you can simply ask `cargo` to download and build this source code for you:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cargo install garage
|
git tag # List available tags
|
||||||
|
git checkout v0.8.0 # Change v0.8.0 with the version you wish to build
|
||||||
```
|
```
|
||||||
|
|
||||||
That's all, `garage` should be in `$HOME/.cargo/bin`.
|
Otherwise you will be building a developpement build from the `main` branch
|
||||||
|
that includes all of the changes to be released in the next version.
|
||||||
|
Be careful that such a build might be unstable or contain bugs,
|
||||||
|
and could be incompatible with nodes that run stable versions of Garage.
|
||||||
|
|
||||||
You can add this folder to your `$PATH` or copy the binary somewhere else on your system.
|
Finally, build Garage with the following command:
|
||||||
For instance:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
sudo cp $HOME/.cargo/bin/garage /usr/local/bin/garage
|
cargo build --release
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The binary built this way can now be found in `target/release/garage`.
|
||||||
|
You may simply copy this binary to somewhere in your `$PATH` in order to
|
||||||
|
have the `garage` command available in your shell, for instance:
|
||||||
|
|
||||||
## Selecting features to activate in your build
|
```bash
|
||||||
|
sudo cp target/release/garage /usr/local/bin/garage
|
||||||
|
```
|
||||||
|
|
||||||
Garage supports a number of compilation options in the form of Cargo features,
|
If you are planning to develop Garage,
|
||||||
|
you might be interested in producing debug builds, which compile faster but run slower:
|
||||||
|
this can be done by removing the `--release` flag, and the resulting build can then
|
||||||
|
be found in `target/debug/garage`.
|
||||||
|
|
||||||
|
## List of available Cargo feature flags
|
||||||
|
|
||||||
|
Garage supports a number of compilation options in the form of Cargo feature flags,
|
||||||
which can be used to provide builds adapted to your system and your use case.
|
which can be used to provide builds adapted to your system and your use case.
|
||||||
The following features are available:
|
To produce a build with a given set of features, invoke the `cargo build` command
|
||||||
|
as follows:
|
||||||
|
|
||||||
| Feature | Enabled | Description |
|
```bash
|
||||||
| ------- | ------- | ----------- |
|
# This will build the default feature set plus feature1, feature2 and feature3
|
||||||
| `bundled-libs` | BY DEFAULT | Use bundled version of sqlite3, zstd, lmdb and libsodium |
|
cargo build --release --features feature1,feature2,feature3
|
||||||
| `system-libs` | optional | Use system version of sqlite3, zstd, lmdb and libsodium if available (exclusive with `bundled-libs`, build using `cargo build --no-default-features --features system-libs`) |
|
# This will build ONLY feature1, feature2 and feature3
|
||||||
| `k2v` | optional | Enable the experimental K2V API (if used, all nodes on your Garage cluster must have it enabled as well) |
|
cargo build --release --no-default-features \
|
||||||
| `kubernetes-discovery` | optional | Enable automatic registration and discovery of cluster nodes through the Kubernetes API |
|
--features feature1,feature2,feature3
|
||||||
| `metrics` | BY DEFAULT | Enable collection of metrics in Prometheus format on the admin API |
|
```
|
||||||
|
|
||||||
|
The following feature flags are available in v0.8.0:
|
||||||
|
|
||||||
|
| Feature flag | Enabled | Description |
|
||||||
|
| ------------ | ------- | ----------- |
|
||||||
|
| `bundled-libs` | *by default* | Use bundled version of sqlite3, zstd, lmdb and libsodium |
|
||||||
|
| `system-libs` | optional | Use system version of sqlite3, zstd, lmdb and libsodium<br>if available (exclusive with `bundled-libs`, build using<br>`cargo build --no-default-features --features system-libs`) |
|
||||||
|
| `k2v` | optional | Enable the experimental K2V API (if used, all nodes on your<br>Garage cluster must have it enabled as well) |
|
||||||
|
| `kubernetes-discovery` | optional | Enable automatic registration and discovery<br>of cluster nodes through the Kubernetes API |
|
||||||
|
| `metrics` | *by default* | Enable collection of metrics in Prometheus format on the admin API |
|
||||||
| `telemetry-otlp` | optional | Enable collection of execution traces using OpenTelemetry |
|
| `telemetry-otlp` | optional | Enable collection of execution traces using OpenTelemetry |
|
||||||
| `sled` | BY DEFAULT | Enable using Sled to store Garage's metadata |
|
| `sled` | *by default* | Enable using Sled to store Garage's metadata |
|
||||||
| `lmdb` | optional | Enable using LMDB to store Garage's metadata |
|
| `lmdb` | optional | Enable using LMDB to store Garage's metadata |
|
||||||
| `sqlite` | optional | Enable using Sqlite3 to store Garage's metadata |
|
| `sqlite` | optional | Enable using Sqlite3 to store Garage's metadata |
|
||||||
|
|
|
@ -26,6 +26,11 @@ a storage plan that best exploits the available storage capacity while satisfyin
|
||||||
To learn more about geo-distributed Garage clusters,
|
To learn more about geo-distributed Garage clusters,
|
||||||
read our documentation on [setting up a real-world deployment](@/documentation/cookbook/real-world.md).
|
read our documentation on [setting up a real-world deployment](@/documentation/cookbook/real-world.md).
|
||||||
|
|
||||||
|
### Standalone/self-contained
|
||||||
|
|
||||||
|
Garage is extremely simple to deploy, and does not depend on any external service to run.
|
||||||
|
This makes setting up and administering storage clusters, we hope, as easy as it could be.
|
||||||
|
|
||||||
### Flexible topology
|
### Flexible topology
|
||||||
|
|
||||||
A Garage cluster can very easily evolve over time, as storage nodes are added or removed.
|
A Garage cluster can very easily evolve over time, as storage nodes are added or removed.
|
||||||
|
@ -46,7 +51,7 @@ This is particularly usefull when nodes are far from one another and talk to one
|
||||||
### Several replication modes
|
### Several replication modes
|
||||||
|
|
||||||
Garage supports a variety of replication modes, with 1 copy, 2 copies or 3 copies of your data,
|
Garage supports a variety of replication modes, with 1 copy, 2 copies or 3 copies of your data,
|
||||||
and with various levels of consistency.
|
and with various levels of consistency, in order to adapt to a variety of usage scenarios.
|
||||||
Read our reference page on [supported replication modes](@/documentation/reference-manual/configuration.md#replication-mode)
|
Read our reference page on [supported replication modes](@/documentation/reference-manual/configuration.md#replication-mode)
|
||||||
to select the replication mode best suited to your use case (hint: in most cases, `replication_mode = "3"` is what you want).
|
to select the replication mode best suited to your use case (hint: in most cases, `replication_mode = "3"` is what you want).
|
||||||
|
|
||||||
|
@ -54,32 +59,67 @@ to select the replication mode best suited to your use case (hint: in most cases
|
||||||
|
|
||||||
A storage bucket can easily be configured to be served directly by Garage as a static web site.
|
A storage bucket can easily be configured to be served directly by Garage as a static web site.
|
||||||
Domain names for multiple websites directly map to bucket names, making it easy to build
|
Domain names for multiple websites directly map to bucket names, making it easy to build
|
||||||
a platform for your user's to autonomously build and host their websites over Garage.
|
a platform for your users to autonomously build and host their websites over Garage.
|
||||||
Surprisingly, none of the other alternative S3 implementations we surveyed (such as Minio
|
Surprisingly, none of the other alternative S3 implementations we surveyed (such as Minio
|
||||||
or CEPH) support publishing static websites from S3 buckets, a feature that is however
|
or CEPH) support publishing static websites from S3 buckets, a feature that is however
|
||||||
directly inherited from S3 on AWS.
|
directly inherited from S3 on AWS.
|
||||||
|
Read more on our [dedicated documentation page](@/documentation/cookbook/exposing-websites.md).
|
||||||
|
|
||||||
### Bucket names as aliases
|
### Bucket names as aliases
|
||||||
|
|
||||||
- the same bucket may have multiple names (useful when exposing websites for example)
|
In Garage, a bucket may have several names, known as aliases.
|
||||||
|
Aliases can easily be added and removed on demand:
|
||||||
|
this allows to easily rename buckets if needed
|
||||||
|
without having to copy all of their content, something that cannot be done on AWS.
|
||||||
|
For buckets served as static websites, having multiple aliases for a bucket can allow
|
||||||
|
exposing the same content under different domain names.
|
||||||
|
|
||||||
- bucket renaming is possible
|
Garage also supports bucket aliases which are local to a single user:
|
||||||
|
this allows different users to have different buckets with the same name, thus avoiding naming collisions.
|
||||||
|
This can be helpfull for instance if you want to write an application that creates per-user buckets with always the same name.
|
||||||
|
|
||||||
- Scoped buckets: 2 users can have a different bucket with the same name -> avoid collision. Helpful if you want to write an application that creates per-user bucket always with the same name.
|
This feature is totally invisible to S3 clients and does not break compatibility with AWS.
|
||||||
|
|
||||||
### Standalone/self contained
|
|
||||||
|
|
||||||
|
|
||||||
### Integration with Kubernetes and Nomad
|
|
||||||
|
|
||||||
Many node discovery methods: Kubernetes integration, Nomad integration through Consul
|
|
||||||
|
|
||||||
### Support for changing IP addresses
|
|
||||||
|
|
||||||
(as long as all nodes don't change their IP at the same time)
|
|
||||||
|
|
||||||
### Cluster administration API
|
### Cluster administration API
|
||||||
|
|
||||||
|
Garage provides a fully-fledged REST API to administer your cluster programatically.
|
||||||
|
Functionnality included in the admin API include: setting up and monitoring
|
||||||
|
cluster nodes, managing access credentials, and managing storage buckets and bucket aliases.
|
||||||
|
A full reference of the administration API is available [here](@/documentation/reference-manual/admin-api.md).
|
||||||
|
|
||||||
### Metrics and traces
|
### Metrics and traces
|
||||||
|
|
||||||
### (experimental) K2V API
|
Garage makes some internal metrics available in the Prometheus data format,
|
||||||
|
which allows you to build interactive dashboards to visualize the load and internal state of your storage cluster.
|
||||||
|
|
||||||
|
For developpers and performance-savvy administrators,
|
||||||
|
Garage also supports exporting traces of what it does internally in OpenTelemetry format.
|
||||||
|
This allows to monitor the time spent at various steps of the processing of requests,
|
||||||
|
in order to detect potential performance bottlenecks.
|
||||||
|
|
||||||
|
### Kubernetes and Nomad integrations
|
||||||
|
|
||||||
|
Garage can automatically discover other nodes in the cluster thanks to integration
|
||||||
|
with orchestrators such as Kubernetes and Nomad (when used with Consul).
|
||||||
|
This eases the configuration of your cluster as it removes one step where nodes need
|
||||||
|
to be manually connected to one another.
|
||||||
|
|
||||||
|
### Support for changing IP addresses
|
||||||
|
|
||||||
|
As long as all of your nodes don't thange their IP address at the same time,
|
||||||
|
Garage should be able to tolerate nodes with changing/dynamic IP addresses,
|
||||||
|
as nodes will regularly exchange the IP addresses of their peers and try to
|
||||||
|
reconnect using newer addresses when existing connections are broken.
|
||||||
|
|
||||||
|
### K2V API (experimental)
|
||||||
|
|
||||||
|
As part of an ongoing research project, Garage can expose an experimental key/value storage API called K2V.
|
||||||
|
K2V is made for the storage and retrieval of many small key/value pairs that need to be processed in bulk.
|
||||||
|
This completes the S3 API with an alternative that can be used to easily store and access metadata
|
||||||
|
related to objects stored in an S3 bucket.
|
||||||
|
|
||||||
|
In the context of our research project, [Aérogramme](https://aerogramme.deuxfleurs.fr),
|
||||||
|
K2V is used to provide metadata and log storage for operations on encrypted e-mail storage.
|
||||||
|
|
||||||
|
Learn more on the specification of K2V [here](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/k2v/doc/drafts/k2v-spec.md)
|
||||||
|
and on how to enable it in Garage [here](@/documentation/reference-manual/k2v.md).
|
||||||
|
|
Loading…
Reference in a new issue