Support STREAMING-AWS4-HMAC-SHA256-PAYLOAD #64

New issue

Closed

opened 2021-04-28 09:01:42 +00:00 by quentin · 2 comments

quentin commented

2021-04-28 09:01:42 +00:00

Owner

A first attempt to support STREAMING-AWS4-HMAC-SHA256-PAYLOAD has been done in dcfc32c but had to be reverted as it corrupted content.

It could be interesting to provide a correct implementation of this feature in Garage. The reference can be accessed here: https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html

A first attempt to support `STREAMING-AWS4-HMAC-SHA256-PAYLOAD` has been done in dcfc32c but had to be reverted as it corrupted content. It could be interesting to provide a correct implementation of this feature in Garage. The reference can be accessed here: https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html

quentin added the

kind

improvement

label 2021-04-28 09:01:43 +00:00

lx added

and removed

labels 2021-04-28 09:03:18 +00:00

lx commented

2021-04-28 09:05:53 +00:00

Owner

This possibly implies a small reorganization of payload verification code that uses different schemes (checksums, signatures, etc)

quentin commented

2021-04-30 16:59:00 +00:00

Author

Owner

After some investigation in the code of minio-go, it appears that STREAMING-AWS4-HMAC-SHA256-PAYLOAD is only used when the connection is not secured (ie. plain http) while UNSIGNED-PAYLOAD is used when the connection is secured (ie. https):

	switch {
      // [...]
	case metadata.objectName != "" && metadata.queryValues == nil && method == http.MethodPut && metadata.customHeader.Get("X-Amz-Copy-Source") == "" && !c.secure:
		// Streaming signature is used by default for a PUT object request. Additionally we also
		// look if the initialized client is secure, if yes then we don't need to perform
		// streaming signature.
		req = signer.StreamingSignV4(req, accessKeyID,
			secretAccessKey, sessionToken, location, metadata.contentLength, time.Now().UTC())
	default:
		// Set sha256 sum for signature calculation only with signature version '4'.
		shaHeader := unsignedPayload
		if metadata.contentSHA256Hex != "" {
			shaHeader = metadata.contentSHA256Hex
		}
		req.Header.Set("X-Amz-Content-Sha256", shaHeader)

		// Add signature version '4' authorization header.
		req = signer.SignV4(*req, accessKeyID, secretAccessKey, sessionToken, location)
	}

7f05f7e977/api.go (L784-L807)

After some investigation in the code of minio-go, it appears that STREAMING-AWS4-HMAC-SHA256-PAYLOAD is only used when the connection is not secured (ie. plain http) while `UNSIGNED-PAYLOAD` is used when the connection is secured (ie. https): ```go switch { // [...] case metadata.objectName != "" && metadata.queryValues == nil && method == http.MethodPut && metadata.customHeader.Get("X-Amz-Copy-Source") == "" && !c.secure: // Streaming signature is used by default for a PUT object request. Additionally we also // look if the initialized client is secure, if yes then we don't need to perform // streaming signature. req = signer.StreamingSignV4(req, accessKeyID, secretAccessKey, sessionToken, location, metadata.contentLength, time.Now().UTC()) default: // Set sha256 sum for signature calculation only with signature version '4'. shaHeader := unsignedPayload if metadata.contentSHA256Hex != "" { shaHeader = metadata.contentSHA256Hex } req.Header.Set("X-Amz-Content-Sha256", shaHeader) // Add signature version '4' authorization header. req = signer.SignV4(*req, accessKeyID, secretAccessKey, sessionToken, location) } ``` https://github.com/minio/minio-go/blob/7f05f7e9774ec1860922ce5bf2f6717faf8677aa/api.go#L784-L807

👍 1

quentin referenced this issue

2021-04-30 20:31:22 +00:00

Add more clients in the test #65

lx referenced this issue from a commit

2021-05-03 15:36:08 +00:00

Remove STREAMING-AWS4-HMAC-SHA256-PAYLOAD (see #64)