Metadata keys are returned with weird casing (instead of all-lowercase) #844
Labels
No labels
action
check-aws
action
discussion-needed
action
for-external-contributors
action
for-newcomers
action
more-info-needed
action
need-funding
action
triage-required
kind
correctness
kind
ideas
kind
improvement
kind
performance
kind
testing
kind
usability
kind
wrong-behavior
prio
critical
prio
low
scope
admin-api
scope
background-healing
scope
build
scope
documentation
scope
k8s
scope
layout
scope
metadata
scope
ops
scope
rpc
scope
s3-api
scope
security
scope
telemetry
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: Deuxfleurs/garage#844
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
I'm getting a
Invalid Signature
response when sending aCopyObject
request from boto3 (specifically, when running this zulip migration:7c07ea86ee/zerver/migrations/0509_fix_emoji_metadata.py (L77-L82)
).Trace logs:
Update: traced this down to the following difference in the hashed canonical request:
boto3 signed it as
This comes from there being duplicate keys in the metadata collection, one with an uppercase first letter, and one with a lowercase one.
From what I can tell, garage seems to be capitalizing the first letter of metadata keys, as zulip is sending them all-lowercase (and is expecting to receive them as such). AWS says that these should be case-insensitive, and all-lowercase: https://docs.aws.amazon.com/codeguru/detector-library/java/s3-object-user-metadata-key-case-sensitivity/
Invalid signature on CopyObject request sent from boto3to Metadata keys are returned with weird casing (instead of all-lowercase)I think the ideal solution would be to return metadata keys all-lowercase, or at least preserve the casing that's sent. The current implementation seems to break things.
Custom object headers are sent here during a GET/HEAD:
https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/src/api/s3/get.rs#L66-L83
Model to store them is defined here:
https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/src/model/s3/object_table.rs#L267-L274
They are persisted here durig a PUT:
https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/src/api/s3/put.rs#L72-L75
More especially, the persisted
HeaderList
is computed from the HTTP framework headers in theget_headers()
function: https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/src/api/s3/put.rs#L621-L629IMO this one is a good candidate to put a
to_lowercase
call.