+++ title = "Browsing tools" weight = 20 +++ Browsing tools allow you to query the S3 API without too many abstractions. These tools are particularly suitable for debug, backups, website deployments or any scripted task that need to handle data. | Name | Status | Note | |------|--------|------| | [Minio client](#minio-client) | ✅ | Recommended | | [AWS CLI](#aws-cli) | ✅ | Recommended | | [rclone](#rclone) | ✅ | | | [s3cmd](#s3cmd) | ✅ | | | [s5cmd](#s5cmd) | ✅ | | | [(Cyber)duck](#cyberduck) | ✅ | | | [WinSCP (libs3)](#winscp) | ✅ | CLI instructions only | | [sftpgo](#sftpgo) | ✅ | | ## Minio client Use the following command to set an "alias", i.e. define a new S3 server to be used by the Minio client: ```bash mc alias set \ garage \ \ \ \ --api S3v4 ``` Remember that `mc` is sometimes called `mcli` (such as on Arch Linux), to avoid conflicts with Midnight Commander. Some commands: ```bash # list buckets mc ls garage/ # list objets in a bucket mc ls garage/my_files # copy from your filesystem to garage mc cp /proc/cpuinfo garage/my_files/cpuinfo.txt # copy from garage to your filesystem mc cp garage/my_files/cpuinfo.txt /tmp/cpuinfo.txt # mirror a folder from your filesystem to garage mc mirror --overwrite ./book garage/garagehq.deuxfleurs.fr ``` ## AWS CLI Create a file named `~/.aws/credentials` and put: ```toml [default] aws_access_key_id=xxxx aws_secret_access_key=xxxx ``` Then a file named `~/.aws/config` and put: ```toml [default] region=garage ``` Now, supposing Garage is listening on `http://127.0.0.1:3900`, you can list your buckets with: ```bash aws --endpoint-url http://127.0.0.1:3900 s3 ls ``` Passing the `--endpoint-url` parameter to each command is annoying but AWS developers do not provide a corresponding configuration entry. As a workaround, you can redefine the aws command by editing the file `~/.bashrc`: ``` function aws { command aws --endpoint-url http://127.0.0.1:3900 $@ ; } ``` *Do not forget to run `source ~/.bashrc` or to start a new terminal before running the next commands.* Now you can simply run: ```bash # list buckets aws s3 ls # list objects of a bucket aws s3 ls s3://my_files # copy from your filesystem to garage aws s3 cp /proc/cpuinfo s3://my_files/cpuinfo.txt # copy from garage to your filesystem aws s3 cp s3/my_files/cpuinfo.txt /tmp/cpuinfo.txt ``` ## `rclone` `rclone` can be configured using the interactive assistant invoked using `rclone config`. You can also configure `rclone` by writing directly its configuration file. Here is a template `rclone.ini` configuration file (mine is located at `~/.config/rclone/rclone.conf`): ```ini [garage] type = s3 provider = Other env_auth = false access_key_id = secret_access_key = region = endpoint = force_path_style = true acl = private bucket_acl = private ``` Now you can run: ```bash # list buckets rclone lsd garage: # list objects of a bucket aggregated in directories rclone lsd garage:my-bucket # copy from your filesystem to garage echo hello world > /tmp/hello.txt rclone copy /tmp/hello.txt garage:my-bucket/ # copy from garage to your filesystem rclone copy garage:quentin.divers/hello.txt . # see all available subcommands rclone help ``` **Advice with rclone:** use the `--fast-list` option when accessing buckets with large amounts of objects. This will tremendously accelerate operations such as `rclone sync` or `rclone ncdu` by reducing the number of ListObjects calls that are made. ## `s3cmd` Here is a template for the `s3cmd.cfg` file to talk with Garage: ```ini [default] access_key = secret_key = host_base = host_bucket = use_https = ``` And use it as follow: ```bash # List buckets s3cmd ls # s3cmd objects inside a bucket s3cmd ls s3://my-bucket # copy from your filesystem to garage echo hello world > /tmp/hello.txt s3cmd put /tmp/hello.txt s3://my-bucket/ # copy from garage to your filesystem s3cmd get s3://my-bucket/hello.txt hello.txt ``` ## `s5cmd` Configure a credentials file as follows: ```bash export AWS_ACCESS_KEY_ID=GK... export AWS_SECRET_ACCESS_KEY= export AWS_DEFAULT_REGION='garage' export AWS_ENDPOINT='http://localhost:3900' ``` After adding these environment variables in your shell, `s5cmd` can be used with: ```bash s5cmd --endpoint-url=$AWS_ENDPOINT ls ``` See its usage output for other commands available. ## Cyberduck & duck {#cyberduck} Both Cyberduck (the GUI) and duck (the CLI) have a concept of "Connection Profiles" that contain some presets for a specific provider. Within Cyberduck, a [Garage connection profile](https://docs.cyberduck.io/protocols/s3/garage/) is available within the `Preferences -> Profiles` section. This can enabled and then connections to Garage may be configured. ### Instuctions for the CLI To configure duck (Cyberduck's CLI tool), start by creating its folder hierarchy: ``` mkdir -p ~/.duck/profiles/ ``` Then, save the connection profile for Garage in `~/.duck/profiles/garage.cyberduckprofile`. To set your credentials in `~/.duck/credentials`, use the following commands to generate the appropriate string: ```bash export AWS_ACCESS_KEY_ID="GK..." export AWS_SECRET_ACCESS_KEY="..." export HOST="s3.garage.localhost" export PORT="4443" export PROTOCOL="https" cat > ~/.duck/credentials <