Version 0.93 breaks with x-amx-content-sha256 bad header?

sansans commented

2024-03-12 06:50:24 +00:00

Using an R library (https://github.com/sansans/minio.s3 forked) to get directly access objects on bucket, previously on 0.91 works . Read documentation that header enforcement? Does this mean require additional headers?

List of 4
$ Code : chr "InvalidRequest"
$ Message : chr "Bad request: Header x-amz-content-sha256 should be signed"
$ Resource: chr "/iistore/test/7dayCovid.csv"
$ Region : chr "garage"

attr(, "headers")=List of 3
..$ content-type : chr "application/xml"
..$ content-length: chr "229"
..$ date : chr "Tue, 12 Mar 2024 05:42:14 GMT"
..- attr(, "class")= chr [1:2] "insensitive" "list"
attr(*, "class")= chr "aws_error"
attr(*, "request_canonical")= chr "GET\n/iistore/test/7day.csv\n\nhost:\nx-amz-date:20240312T054214Z\n\nhost;x-amz-date\ne3"| truncated
attr(*, "request_string_to_sign")= chr "AWS4-HMAC-SHA256\n20240312T054214Z\n20240312/garage/s3/aws4_request\n8fb418147549af04f6e7709b46636f84b08d85b494"| truncated
attr(*, "request_signature")= chr "AWS4-HMAC-SHA256 Credential=xxxx/garage/s3/aws4_request,SignedHeaders=host;x-amz"| truncated
NULL

Using an R library (https://github.com/sansans/minio.s3 forked) to get directly access objects on bucket, previously on 0.91 works . Read documentation that header enforcement? Does this mean require additional headers? List of 4 $ Code : chr "InvalidRequest" $ Message : chr "Bad request: Header `x-amz-content-sha256` should be signed" $ Resource: chr "/iistore/test/7dayCovid.csv" $ Region : chr "garage" - attr(*, "headers")=List of 3 ..$ content-type : chr "application/xml" ..$ content-length: chr "229" ..$ date : chr "Tue, 12 Mar 2024 05:42:14 GMT" ..- attr(*, "class")= chr [1:2] "insensitive" "list" - attr(*, "class")= chr "aws_error" - attr(*, "request_canonical")= chr "GET\n/iistore/test/7day.csv\n\nhost:\nx-amz-date:20240312T054214Z\n\nhost;x-amz-date\ne3"| __truncated__ - attr(*, "request_string_to_sign")= chr "AWS4-HMAC-SHA256\n20240312T054214Z\n20240312/garage/s3/aws4_request\n8fb418147549af04f6e7709b46636f84b08d85b494"| __truncated__ - attr(*, "request_signature")= chr "AWS4-HMAC-SHA256 Credential=xxxx/garage/s3/aws4_request,SignedHeaders=host;x-amz"| __truncated__ NULL

lx added the

Bug

S3 Compatibility

labels 2024-03-12 10:38:40 +00:00

lx added this to the v1.0 milestone 2024-03-12 10:38:43 +00:00

lx commented

2024-03-12 10:38:50 +00:00

Owner

Thanks, will look into it

lx commented

2024-03-12 10:44:22 +00:00

Owner

I have a feeling that this is a bug in your S3 client library, as Amazon explicitly says that the x-amz-content-sha256 should be included in the signature calculation. See examples here: https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html

However, if we can confirm that this library works well on Amazon's implementation of S3, it means that S3 is actually more relaxed about this, and we should be too in our implementation. It would not surprise me so much, because the value of the x-amz-content-sha256 header is also included as the last line of the canonical request, so it is part of the signature calculation in all cases.

@sansans , would you be willing to open a free AWS account and check that your library works well on official Amazon S3?

I have a feeling that this is a bug in your S3 client library, as Amazon explicitly says that the `x-amz-content-sha256` should be included in the signature calculation. See examples here: https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html However, if we can confirm that this library works well on Amazon's implementation of S3, it means that S3 is actually more relaxed about this, and we should be too in our implementation. It would not surprise me so much, because the value of the x-amz-content-sha256 header is also included as the last line of the canonical request, so it is part of the signature calculation in all cases. @sansans , would you be willing to open a free AWS account and check that your library works well on official Amazon S3?

sansans commented

2024-03-13 09:22:26 +00:00

Author

Ok will do that, I'll try and will see the results.

lx added

Check AWS

and removed

Bug

labels 2024-03-15 13:19:26 +00:00

lx removed this from the v1.0 milestone 2024-03-15 13:19:32 +00:00

sansans commented

2024-03-22 04:38:20 +00:00

Author

updates, I've tested with an Amazon hosted S3 bucket using the library. Works fine. I guess as you suspected it's relaxed about the header. Also tested with Minio self hosted, no issues as well.

Version 0.93 breaks with x-amx-content-sha256 bad header? #770