Empty Multipart Uploads Fail #759

New Issue

Open

opened 2024-03-04 14:27:57 +00:00 by gkleen · 2 comments

gkleen commented

2024-03-04 14:27:57 +00:00

Trying to use pgbackrest 2.10 against garage it occasionally tries upload empty files and seems to use the multipart upload api unconditionally.

This fails on src/api/s3/multipart.rs:99, producing:

<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRequest</Code><Message>Bad request: Empty body</Message><Resource>…</Resource><Region>the39</Region></Error>

After discussion on the matrix channel this was deemed "a compatibility bug in Garage".

Should I formulate some sort of minimal demonstrating example?

Trying to use pgbackrest 2.10 against garage it occasionally tries upload empty files and seems to use the multipart upload api unconditionally. This fails on `src/api/s3/multipart.rs:99`, producing: ```xml <?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidRequest</Code><Message>Bad request: Empty body</Message><Resource>…</Resource><Region>the39</Region></Error> ``` After discussion on the matrix channel this was deemed "a compatibility bug in Garage". Should I formulate some sort of minimal demonstrating example?

lx added the

S3 Compatibility

label 2024-03-04 15:05:44 +00:00

lx commented

2024-03-04 15:07:41 +00:00

Owner

Next steps:

Confirm that AWS allows to upload an empty object using UploadPart
Determine how to fix this internally. Normally very small objects are stored inline, so maybe CompleteMultipartUpload should detect the case of an empty object and convert it to inline? Probably even skip inserting a block in the version, we don't want to create a zero-length block.
Implement fix

Next steps: - [ ] Confirm that AWS allows to upload an empty object using UploadPart - [ ] Determine how to fix this internally. Normally very small objects are stored inline, so maybe CompleteMultipartUpload should detect the case of an empty object and convert it to inline? Probably even skip inserting a block in the version, we don't want to create a zero-length block. - [ ] Implement fix

lx commented

2024-03-04 15:08:44 +00:00

Owner

Should I formulate some sort of minimal demonstrating example?

If you'd like, it would be great to have a test case in the form of shell comands, for instance using the aws s3api tool. If you can demonstrate a set of commands that work on AWS and not on Garage, that would be perfect.

> Should I formulate some sort of minimal demonstrating example? If you'd like, it would be great to have a test case in the form of shell comands, for instance using the [aws s3api](https://docs.aws.amazon.com/cli/latest/reference/s3api/) tool. If you can demonstrate a set of commands that work on AWS and not on Garage, that would be perfect.

lx added the

Check AWS

label 2024-03-15 13:20:06 +00:00