Support STREAMING-AWS4-HMAC-SHA256-PAYLOAD #64

Closed
opened 1 year ago by quentin · 2 comments
quentin commented 1 year ago
Owner

A first attempt to support STREAMING-AWS4-HMAC-SHA256-PAYLOAD has been done in dcfc32c but had to be reverted as it corrupted content.

It could be interesting to provide a correct implementation of this feature in Garage. The reference can be accessed here: https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html

A first attempt to support `STREAMING-AWS4-HMAC-SHA256-PAYLOAD` has been done in dcfc32c but had to be reverted as it corrupted content. It could be interesting to provide a correct implementation of this feature in Garage. The reference can be accessed here: https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html
quentin added the
Improvement
label 1 year ago
lx added
S3 Compatibility
and removed
Improvement
labels 1 year ago
lx commented 1 year ago
Owner

This possibly implies a small reorganization of payload verification code that uses different schemes (checksums, signatures, etc)

This possibly implies a small reorganization of payload verification code that uses different schemes (checksums, signatures, etc)
Poster
Owner

After some investigation in the code of minio-go, it appears that STREAMING-AWS4-HMAC-SHA256-PAYLOAD is only used when the connection is not secured (ie. plain http) while UNSIGNED-PAYLOAD is used when the connection is secured (ie. https):

	switch {
      // [...]
	case metadata.objectName != "" && metadata.queryValues == nil && method == http.MethodPut && metadata.customHeader.Get("X-Amz-Copy-Source") == "" && !c.secure:
		// Streaming signature is used by default for a PUT object request. Additionally we also
		// look if the initialized client is secure, if yes then we don't need to perform
		// streaming signature.
		req = signer.StreamingSignV4(req, accessKeyID,
			secretAccessKey, sessionToken, location, metadata.contentLength, time.Now().UTC())
	default:
		// Set sha256 sum for signature calculation only with signature version '4'.
		shaHeader := unsignedPayload
		if metadata.contentSHA256Hex != "" {
			shaHeader = metadata.contentSHA256Hex
		}
		req.Header.Set("X-Amz-Content-Sha256", shaHeader)

		// Add signature version '4' authorization header.
		req = signer.SignV4(*req, accessKeyID, secretAccessKey, sessionToken, location)
	}

7f05f7e977/api.go (L784-L807)

After some investigation in the code of minio-go, it appears that STREAMING-AWS4-HMAC-SHA256-PAYLOAD is only used when the connection is not secured (ie. plain http) while `UNSIGNED-PAYLOAD` is used when the connection is secured (ie. https): ```go switch { // [...] case metadata.objectName != "" && metadata.queryValues == nil && method == http.MethodPut && metadata.customHeader.Get("X-Amz-Copy-Source") == "" && !c.secure: // Streaming signature is used by default for a PUT object request. Additionally we also // look if the initialized client is secure, if yes then we don't need to perform // streaming signature. req = signer.StreamingSignV4(req, accessKeyID, secretAccessKey, sessionToken, location, metadata.contentLength, time.Now().UTC()) default: // Set sha256 sum for signature calculation only with signature version '4'. shaHeader := unsignedPayload if metadata.contentSHA256Hex != "" { shaHeader = metadata.contentSHA256Hex } req.Header.Set("X-Amz-Content-Sha256", shaHeader) // Add signature version '4' authorization header. req = signer.SignV4(*req, accessKeyID, secretAccessKey, sessionToken, location) } ``` https://github.com/minio/minio-go/blob/7f05f7e9774ec1860922ce5bf2f6717faf8677aa/api.go#L784-L807
KokaKiwi was assigned by quentin 11 months ago
lx closed this issue 8 months ago
Sign in to join this conversation.
No Milestone
No Assignees
2 Participants
Notifications
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Loading…
There is no content yet.