path style looks broken in 0.9.0 #673

New issue

Closed

opened 2023-11-26 13:00:03 +00:00 by schmitch · 4 comments

schmitch commented

2023-11-26 13:00:03 +00:00

when doing a request to 0.9.0 in path style mode it will fail if the request is like that:

GET /postgres-backup?delimiter=%2F&list-type=2&prefix=spilo%2Fenvisia-postgres%2F762e8a5b-4df2-427b-9e77-6c9eb06bd16b%2Fwal%2F14%2Fbasebackups_005%2F HTTP/1.1

it fails with:

<?xml version="1.0" encoding="UTF-8"?><Error><Code>NoSuchBucket</Code><Message>Bucket not found: garage</Message><Resource>/envisia-loki-pg-backup</Resource><Region>garage</Region></Error>

what works if there is a / at the end:

GET /postgres-backup/?delimiter=%2F&list-type=2&prefix=spilo%2Fenvisia-postgres%2F762e8a5b-4df2-427b-9e77-6c9eb06bd16b%2Fwal%2F14%2Fbasebackups_005%2F HTTP/1.1

in fact the same call worked on 0.8.x. unfortunatly wal-g does not add the trailing / and thus would fail in cases like that.

actually wal-g uses aws-sdk-go/1.44.7 maybe that is in itself a problem.

when doing a request to 0.9.0 in path style mode it will fail if the request is like that: ``` GET /postgres-backup?delimiter=%2F&list-type=2&prefix=spilo%2Fenvisia-postgres%2F762e8a5b-4df2-427b-9e77-6c9eb06bd16b%2Fwal%2F14%2Fbasebackups_005%2F HTTP/1.1 ``` it fails with: ``` <?xml version="1.0" encoding="UTF-8"?><Error><Code>NoSuchBucket</Code><Message>Bucket not found: garage</Message><Resource>/envisia-loki-pg-backup</Resource><Region>garage</Region></Error> ``` what works if there is a `/` at the end: ``` GET /postgres-backup/?delimiter=%2F&list-type=2&prefix=spilo%2Fenvisia-postgres%2F762e8a5b-4df2-427b-9e77-6c9eb06bd16b%2Fwal%2F14%2Fbasebackups_005%2F HTTP/1.1 ``` in fact the same call worked on 0.8.x. unfortunatly wal-g does not add the trailing `/` and thus would fail in cases like that. actually wal-g uses `aws-sdk-go/1.44.7` maybe that is in itself a problem.

trinity-1686a commented

2023-11-26 16:53:18 +00:00

Owner

if a / is important, the code that would be faulty would probably be in router.rs, but it looks fine, and untouched for a long time.

Are you sure nothing else than that / changes between a successful and an unsuccessful query? What is the Host: header in both cases, and what is the value of s3_api.root_domain (if any) in your configuration file?

if a `/` is important, the code that would be faulty would probably be [in router.rs](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/src/api/s3/router.rs#L313-L327), but it looks fine, and untouched for a long time. Are you sure nothing else than that `/` changes between a successful and an unsuccessful query? What is the `Host:` header in both cases, and what is the value of `s3_api.root_domain` (if any) in your configuration file?

schmitch commented

2023-11-27 09:04:40 +00:00

Author

the one that is not working has the following inside the s3_api.root_domain:

    [s3_api]
    s3_region = "garage"
    api_bind_addr = "[::]:3900"
    root_domain = ".garage.svc.cluster.local"

but I can probably remove the root domain, since it is not in use. maybe thats probably also the problem since the client will go to:

garage.garage.svc.cluster.local really really good catch @trinity-1686a that probably helped, I will try that out later.

The working one has .s3.envisia.net and the clients use garage.garage.svc.cluster.local as well.

the one that is not working has the following inside the `s3_api.root_domain`: ``` [s3_api] s3_region = "garage" api_bind_addr = "[::]:3900" root_domain = ".garage.svc.cluster.local" ``` but I can probably remove the root domain, since it is not in use. maybe thats probably also the problem since the client will go to: `garage.garage.svc.cluster.local` really really good catch @trinity-1686a that probably helped, I will try that out later. The working one has `.s3.envisia.net` and the clients use `garage.garage.svc.cluster.local` as well.

trinity-1686a commented

2023-11-27 09:20:40 +00:00

Owner

with a root_domain of garage.svc.cluster.local, and a query for garage.garage.svc.cluster.local, Garage will remove the suffix and consider that the first "garage" is the bucket name (vhost style). With s3.envisia.net and garage.garage.svc.cluster.local, the suffix mismatches so Garage tries path-style

with a root_domain of `garage.svc.cluster.local`, and a query for `garage.garage.svc.cluster.local`, Garage will remove the suffix and consider that the first "garage" is the bucket name (vhost style). With `s3.envisia.net` and `garage.garage.svc.cluster.local`, the suffix mismatches so Garage tries path-style

schmitch commented

2023-11-29 13:48:27 +00:00

Author

@trinity-1686a thanks! that helped. Now everything works!