Full compatibility with S3 multipart uploads #204
Labels
No Label
AdminAPI
Bug
Check AWS
CI
Correctness
Critical
Documentation
Ideas
Improvement
Low priority
Newcomer
Performance
S3 Compatibility
Testing
Usability
No Milestone
No Assignees
3 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: Deuxfleurs/garage#204
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Task 1 of the NLnet Garage project started in April 2023 consists in fixing this issue. Progress on this task is tracked in #553.
See #197 and #198.
We are missing the following features:
Ability to re-upload a part that has been uploaded, by re-calling UploadPart. It should be silently overwritten:
https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPart.html
Ability to drop some of the uploaded parts by not including them in the CompleteMultipartUpload call
On CompleteMultipartUpload, renumbering of parts with consecutive numbers starting at 1 (necessary so that the
partNumber
parameter in GetObject and HeadObject works correctly).These features can only be implemented by fundamentally reworking the data model of Garage, meaning it's kind of a big change we aren't going to do just now.
For now, since renumbering is not supported, and to ensure that
partNumber
works correctly, the following restriction is implemented:Moreover, the following restrictions apply:
All uploaded parts must be used in CompleteMultipartUpload.
In case one of the parts upload went wrong, you have to abort the multipart upload and start over.
I think this is the most annoying part of this ticket. It's one of the main selling point of multipart upload with higher bandwidth usage. From Amazon documentation :
We observed one of the undesired behaviour in our CI during a nightly build.
Some artefacts failed to upload as a part upload was interrupted and couldn't be resumed by Garage.