Ring not yet ready, read/writes will be lost! #965

New issue

Open

opened 2025-02-19 08:36:22 +00:00 by Tugdualenligne · 2 comments

Tugdualenligne commented

2025-02-19 08:36:22 +00:00

2025-02-19T08:03:08.988976Z WARN garage_rpc::layout::version: Ring not yet ready, read/writes will be lost!
2025-02-19T08:03:08.988981Z WARN garage_rpc::layout::version: Ring not yet ready, read/writes will be lost!
2025-02-19T08:03:08.988987Z WARN garage_rpc::layout::version: Ring not yet ready, read/writes will be lost!
2025-02-19T08:03:08.988995Z INFO garage_table::sync: k2v_index_counter_v2: Adding full sync for ack layout version 0
2025-02-19T08:03:08.989240Z INFO garage_table::sync: bucket_v2: Completed full sync for ack layout version 0
2025-02-19T08:03:08.989345Z INFO garage_table::sync: bucket_alias: Completed full sync for ack layout version 0
2025-02-19T08:03:08.989432Z INFO garage_table::sync: key: Completed full sync for ack layout version 0
2025-02-19T08:03:09.083971Z INFO garage_table::sync: object: Completed full sync for ack layout version 0
2025-02-19T08:03:09.084170Z INFO garage_table::sync: bucket_object_counter: Completed full sync for ack layout version 0
2025-02-19T08:03:09.084284Z INFO garage_table::sync: multipart_upload: Completed full sync for ack layout version 0
2025-02-19T08:03:09.084401Z INFO garage_table::sync: bucket_mpu_counter: Completed full sync for ack layout version 0
2025-02-19T08:03:09.084509Z INFO garage_table::sync: version: Completed full sync for ack layout version 0
2025-02-19T08:03:09.084654Z INFO garage_table::sync: block_ref: Completed full sync for ack layout version 0
2025-02-19T08:03:09.084762Z INFO garage_table::sync: k2v_item: Completed full sync for ack layout version 0
2025-02-19T08:03:09.084856Z INFO garage_table::sync: k2v_index_counter_v2: Completed full sync for ack layout version 0

services:
  garage:
    image: dxflrs/garage:v1.0.1
    container_name: garage-s3
    networks:
      - traefik_proxy
    ports: # fonctionne décommentés
      - "3900:3900" 
      - "3901:3901"
      - "3902:3902"
      - "3903:3903"
      - "3904:3904"
    volumes:
      - /var/lib/docker/volumes/garage-s3/_data/garage.toml:/etc/garage.toml
      - garage-s3_meta:/var/lib/garage/meta
      - /srv/dev-disk-by-uuid-07e5da0a-a3ba-45c9-a55f-59af0e89aa06/backup3:/var/lib/garage/data/backup3
      - /srv/dev-disk-by-uuid-0577ca8c-a79b-42db-bfb3-e3e2eeaf5ed9/backup4:/var/lib/garage/data/backup4
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=traefik_proxy"
      - "traefik.http.routers.garage-s3.rule=Host(`s3b.domain.fr`)"
      - "traefik.http.routers.garage-s3.entrypoints=https"
      - "traefik.http.routers.garage-s3.tls=true"
      - "traefik.http.routers.garage-s3.tls.certresolver=acme"
      - "traefik.http.services.garage-s3.loadbalancer.server.port=3900" # ou 3901
#      - "traefik.http.routers.garage-s3-console.rule=Host(`minio-s3b.domain.fr`)"
#      - "traefik.http.routers.garage-s3-console.entrypoints=https"
#      - "traefik.http.routers.garage-s3-console.tls=true"
#      - "traefik.http.routers.garage-s3-console.tls.certresolver=acme"
#      - "traefik.http.services.garage-s3-console.loadbalancer.server.port=3902"
      - "diun.enable=true"
    restart: unless-stopped

networks:
  default:
  traefik_proxy:
    external: true
    name: traefik_proxy
volumes:
  garage-s3:
#    external: true
  garage-s3_meta:
#    external: true

And the garage.toml is:

metadata_dir = "/var/lib/docker/volumes/garage-s3_meta/_data"
data_dir = [
    { path = "/srv/dev-disk-by-uuid-07e5da0a-a3ba-45c9-a55f-59af0e89aa06/backup3", capacity = "1T" },
#    { path = "/srv/dev-disk-by-uuid-0577ca8c-a79b-42db-bfb3-e3e2eeaf5ed9/backup4", capacity = "1T" },
]
db_engine = "sqlite"
metadata_auto_snapshot_interval = "6h"

replication_mode = "none"

compression_level = 2

rpc_bind_addr = "[::]:3901"
rpc_public_addr = "127.0.0.1:3901"
rpc_secret = "secret_key1"

[s3_api]
s3_region = "berdoues"
api_bind_addr = "[::]:3900"
root_domain = ".domain.fr"

[s3_web]
bind_addr = "[::]:3902"
root_domain = "minio-s3b.domain.fr"
index = "index.html"

[k2v_api]
api_bind_addr = "[::]:3904"

[admin]
api_bind_addr = "[::]:3903"
admin_token = "secret_key2"
metrics_token = "secret_key3"```

HI there, thanks for this S3 software! I'm trying this software: I can create a layout, apply it, create a bucket, create a key, assign it to the bucket, but I still get the first lines of errors below, and each restart of my docker container make me loss all the config / buckets / keys created. I don't understand how to correct that I have Garage as a Docker container, with Traefik as reverse proxy: below is my docker compose 2025-02-19T08:03:08.988976Z WARN garage_rpc::layout::version: Ring not yet ready, read/writes will be lost! 2025-02-19T08:03:08.988981Z WARN garage_rpc::layout::version: Ring not yet ready, read/writes will be lost! 2025-02-19T08:03:08.988987Z WARN garage_rpc::layout::version: Ring not yet ready, read/writes will be lost! 2025-02-19T08:03:08.988995Z INFO garage_table::sync: k2v_index_counter_v2: Adding full sync for ack layout version 0 2025-02-19T08:03:08.989240Z INFO garage_table::sync: bucket_v2: Completed full sync for ack layout version 0 2025-02-19T08:03:08.989345Z INFO garage_table::sync: bucket_alias: Completed full sync for ack layout version 0 2025-02-19T08:03:08.989432Z INFO garage_table::sync: key: Completed full sync for ack layout version 0 2025-02-19T08:03:09.083971Z INFO garage_table::sync: object: Completed full sync for ack layout version 0 2025-02-19T08:03:09.084170Z INFO garage_table::sync: bucket_object_counter: Completed full sync for ack layout version 0 2025-02-19T08:03:09.084284Z INFO garage_table::sync: multipart_upload: Completed full sync for ack layout version 0 2025-02-19T08:03:09.084401Z INFO garage_table::sync: bucket_mpu_counter: Completed full sync for ack layout version 0 2025-02-19T08:03:09.084509Z INFO garage_table::sync: version: Completed full sync for ack layout version 0 2025-02-19T08:03:09.084654Z INFO garage_table::sync: block_ref: Completed full sync for ack layout version 0 2025-02-19T08:03:09.084762Z INFO garage_table::sync: k2v_item: Completed full sync for ack layout version 0 2025-02-19T08:03:09.084856Z INFO garage_table::sync: k2v_index_counter_v2: Completed full sync for ack layout version 0 ``` services: garage: image: dxflrs/garage:v1.0.1 container_name: garage-s3 networks: - traefik_proxy ports: # fonctionne décommentés - "3900:3900" - "3901:3901" - "3902:3902" - "3903:3903" - "3904:3904" volumes: - /var/lib/docker/volumes/garage-s3/_data/garage.toml:/etc/garage.toml - garage-s3_meta:/var/lib/garage/meta - /srv/dev-disk-by-uuid-07e5da0a-a3ba-45c9-a55f-59af0e89aa06/backup3:/var/lib/garage/data/backup3 - /srv/dev-disk-by-uuid-0577ca8c-a79b-42db-bfb3-e3e2eeaf5ed9/backup4:/var/lib/garage/data/backup4 labels: - "traefik.enable=true" - "traefik.docker.network=traefik_proxy" - "traefik.http.routers.garage-s3.rule=Host(`s3b.domain.fr`)" - "traefik.http.routers.garage-s3.entrypoints=https" - "traefik.http.routers.garage-s3.tls=true" - "traefik.http.routers.garage-s3.tls.certresolver=acme" - "traefik.http.services.garage-s3.loadbalancer.server.port=3900" # ou 3901 # - "traefik.http.routers.garage-s3-console.rule=Host(`minio-s3b.domain.fr`)" # - "traefik.http.routers.garage-s3-console.entrypoints=https" # - "traefik.http.routers.garage-s3-console.tls=true" # - "traefik.http.routers.garage-s3-console.tls.certresolver=acme" # - "traefik.http.services.garage-s3-console.loadbalancer.server.port=3902" - "diun.enable=true" restart: unless-stopped networks: default: traefik_proxy: external: true name: traefik_proxy volumes: garage-s3: # external: true garage-s3_meta: # external: true ``` And the garage.toml is: ``` metadata_dir = "/var/lib/docker/volumes/garage-s3_meta/_data" data_dir = [ { path = "/srv/dev-disk-by-uuid-07e5da0a-a3ba-45c9-a55f-59af0e89aa06/backup3", capacity = "1T" }, # { path = "/srv/dev-disk-by-uuid-0577ca8c-a79b-42db-bfb3-e3e2eeaf5ed9/backup4", capacity = "1T" }, ] db_engine = "sqlite" metadata_auto_snapshot_interval = "6h" replication_mode = "none" compression_level = 2 rpc_bind_addr = "[::]:3901" rpc_public_addr = "127.0.0.1:3901" rpc_secret = "secret_key1" [s3_api] s3_region = "berdoues" api_bind_addr = "[::]:3900" root_domain = ".domain.fr" [s3_web] bind_addr = "[::]:3902" root_domain = "minio-s3b.domain.fr" index = "index.html" [k2v_api] api_bind_addr = "[::]:3904" [admin] api_bind_addr = "[::]:3903" admin_token = "secret_key2" metrics_token = "secret_key3"```

lx commented

2025-02-19 09:54:50 +00:00

Owner

Your garage.toml configuration file is wrong, you have to specify data_dir and metadata_dir paths within the Docker container (/var/lib/garage/...) and not on the host system.

Your `garage.toml` configuration file is wrong, you have to specify `data_dir` and `metadata_dir` paths within the Docker container (`/var/lib/garage/...`) and not on the host system.

Tugdualenligne commented

2025-02-20 15:53:59 +00:00

Author

Many thanks Alex, this enabled me to correct a get my Garage S3 up and running! Currently understanding / testing its capacities.

I still have for some reason a strange behavior where not every time I can access my external S3 endpoint from the internet, but I suspect it has more to do with my Traefik proxy (strange though because I have 50 other containers running where I don't have that sort of issue). If I change a setting in my Traefik section of the Garage docker compose, and restart the container, it usually corrects the external connectivity.

Many thanks Alex, this enabled me to correct a get my Garage S3 up and running! Currently understanding / testing its capacities. I still have for some reason a strange behavior where not every time I can access my external S3 endpoint from the internet, but I suspect it has more to do with my Traefik proxy (strange though because I have 50 other containers running where I don't have that sort of issue). If I change a setting in my Traefik section of the Garage docker compose, and restart the container, it usually corrects the external connectivity.

No milestone

No project

No assignees

2 participants

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Deuxfleurs/garage#965

No description provided.

Rows
Columns