If-Match Not Implemented for GET #804

New Issue

Open

opened 2024-04-10 21:50:46 +00:00 by DougReeder · 2 comments

DougReeder commented

2024-04-10 21:50:46 +00:00

Scenario: When an If-Match header is passed to Garage on a GET request, as it would be for a Range Request.

(An object is present at path s3Path and req.get('If-Match') returns an ETag different than the ETag of the existing version.)

Steps (using @aws-sdk/client-s3 in an Express server)

      try {
        const bucketName = calcBucketName(req.params.username);
        const s3Path = calcS3Path(BLOB_PREFIX, isFolderRequest ? req.params[0].slice(0, -1) : req.params[0]);
        let getParam;
        if (req.get('If-None-Match')) {
          getParam = { Bucket: bucketName, Key: s3Path, IfNoneMatch: req.get('If-None-Match') };
        } else if (req.get('If-Match')) {
          getParam = { Bucket: bucketName, Key: s3Path, IfMatch: req.get('If-Match') };
        } else { // unconditional
          getParam = { Bucket: bucketName, Key: s3Path };
        }
        const { Body, ETag, ContentType, ContentLength } = await s3client.send(new GetObjectCommand(getParam));
          res.status(200).set('Content-Length', ContentLength).set('Content-Type', contentType).set('ETag', normalizeETag(ETag));
          return pipeline(Body, res);
      } catch (err) {
        if (['NotFound', 'NoSuchKey', 'Forbidden', 'AccessDenied', '403'].includes(err.name) ||
            err.$metadata?.httpStatusCode === 403) {
            return res.status(404).end(); // Not Found
        } else if (err.name === 'PreconditionFailed') {
          return res.status(412).end();
        } else if (['NotModified', '304'].includes(err.name) || err.$metadata?.httpStatusCode === 304) {
          return res.status(304).end();
        } else if (err.name === 'EndResponseError') {
          res.logNotes.add(err.message);
          res.logLevel = err.logLevel;
          return res.status(err.statusCode).type('text/plain').send(err.message);
        } else {
          return next(Object.assign(err, { status: 502 }));
        }
      }

Expected Result
await s3client.send(new GetObjectCommand(getParam)) throws a PreconditionFailed error.

Actual Result
await s3client.send(new GetObjectCommand(getParam)) returns a ServerResponse with statusCode === 200

### **Scenario**: When an If-Match header is passed to Garage on a GET request, as it would be for a Range Request. (An object is present at path `s3Path` and `req.get('If-Match')` returns an ETag different than the ETag of the existing version.) **Steps** (using @aws-sdk/client-s3 in an Express server) ``` try { const bucketName = calcBucketName(req.params.username); const s3Path = calcS3Path(BLOB_PREFIX, isFolderRequest ? req.params[0].slice(0, -1) : req.params[0]); let getParam; if (req.get('If-None-Match')) { getParam = { Bucket: bucketName, Key: s3Path, IfNoneMatch: req.get('If-None-Match') }; } else if (req.get('If-Match')) { getParam = { Bucket: bucketName, Key: s3Path, IfMatch: req.get('If-Match') }; } else { // unconditional getParam = { Bucket: bucketName, Key: s3Path }; } const { Body, ETag, ContentType, ContentLength } = await s3client.send(new GetObjectCommand(getParam)); res.status(200).set('Content-Length', ContentLength).set('Content-Type', contentType).set('ETag', normalizeETag(ETag)); return pipeline(Body, res); } catch (err) { if (['NotFound', 'NoSuchKey', 'Forbidden', 'AccessDenied', '403'].includes(err.name) || err.$metadata?.httpStatusCode === 403) { return res.status(404).end(); // Not Found } else if (err.name === 'PreconditionFailed') { return res.status(412).end(); } else if (['NotModified', '304'].includes(err.name) || err.$metadata?.httpStatusCode === 304) { return res.status(304).end(); } else if (err.name === 'EndResponseError') { res.logNotes.add(err.message); res.logLevel = err.logLevel; return res.status(err.statusCode).type('text/plain').send(err.message); } else { return next(Object.assign(err, { status: 502 })); } } ``` **Expected Result** `await s3client.send(new GetObjectCommand(getParam))` throws a PreconditionFailed error. **Actual Result** `await s3client.send(new GetObjectCommand(getParam))` returns a ServerResponse with statusCode === 200

lx commented

2024-04-11 11:30:51 +00:00

Owner

I'm having a hard time understanding exactly what is the request that your code makes.

Could you rather set your Garage daemon to debug mode (RUST_LOG=garage_api=debug) and paste the log lines corresponding to the request and its associated response?

Thanks

I'm having a hard time understanding exactly what is the request that your code makes. Could you rather set your Garage daemon to debug mode (`RUST_LOG=garage_api=debug`) and paste the log lines corresponding to the request and its associated response? Thanks

DougReeder commented

2024-04-11 15:11:04 +00:00

Author

2024-04-11 11:00:44 2024-04-11T15:00:44.778140Z  INFO garage_api::generic_server: [::ffff:192.168.65.1]:17202 GET /automated-test-6996524121749715-java.extraordinary.org/remoteStorageBlob/existing/short-story?x-id=GetObject
2024-04-11 11:00:44 2024-04-11T15:00:44.778171Z DEBUG garage_api::generic_server: Request { method: GET, uri: /automated-test-6996524121749715-java.extraordinary.org/remoteStorageBlob/existing/short-story?x-id=GetObject, version: HTTP/1.1, headers: {**"if-match": "\"6kjl35j6365k\"**", "host": "127.0.0.1:3900", "x-amz-user-agent": "aws-sdk-js/3.523.0", "user-agent": "aws-sdk-js/3.523.0 ua/2.0 os/darwin#23.4.0 lang/js md/nodejs#20.10.0 api/s3#3.523.0", "amz-sdk-invocation-id": "7373b9d2-3439-45ac-a605-e4ec6ea28957", "amz-sdk-request": "attempt=1; max=3", "x-amz-date": "20240411T150044Z", "x-amz-content-sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "authorization": "AWS4-HMAC-SHA256 Credential=GK5bec374936673b0b236b1618/20240411/garage/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;host;if-match;x-amz-content-sha256;x-amz-date;x-amz-user-agent, Signature=868d4fca83d2e4672c671fc7bb2a270d98448bb7a198745941c1329df85a5c2d", "connection": "keep-alive"}, body: Body(Empty) }
2024-04-11 11:00:44 2024-04-11T15:00:44.778194Z DEBUG garage_api::s3::router: Received an unknown query parameter: 'x-id'
2024-04-11 11:00:44 2024-04-11T15:00:44.778198Z DEBUG garage_api::generic_server: Endpoint: GetObject
2024-04-11 11:00:44 2024-04-11T15:00:44.778856Z DEBUG garage_api::s3::get: Version meta: ObjectVersionMeta { headers: ObjectVersionHeaders { content_type: "text/plain", other: {} }, size: 6, etag: "a2ec0c77b7bea23455185bcc75535bf7" }
2024-04-11 11:00:44 2024-04-11T15:00:44.778929Z DEBUG garage_api::generic_server: 200 OK {"content-type": "text/plain", "last-modified": "Thu, 11 Apr 2024 15:00:30 GMT", "accept-ranges": "bytes", **"etag": "\"a2ec0c77b7bea23455185bcc75535bf7\""**, "content-length": "6"}

Note that the last line shows that the ETag of the existing object is "a2ec0c77b7bea23455185bcc75535bf7". The request sends the if-match header with value "6kjl35j6365k", so it doesn't match, and the request should return status 412 Precondition Failed

By itself, If-Match on a GET request is rarely useful, but when making a Range request, it's essential that the range come from the same version of the object.

``` 2024-04-11 11:00:44 2024-04-11T15:00:44.778140Z INFO garage_api::generic_server: [::ffff:192.168.65.1]:17202 GET /automated-test-6996524121749715-java.extraordinary.org/remoteStorageBlob/existing/short-story?x-id=GetObject 2024-04-11 11:00:44 2024-04-11T15:00:44.778171Z DEBUG garage_api::generic_server: Request { method: GET, uri: /automated-test-6996524121749715-java.extraordinary.org/remoteStorageBlob/existing/short-story?x-id=GetObject, version: HTTP/1.1, headers: {**"if-match": "\"6kjl35j6365k\"**", "host": "127.0.0.1:3900", "x-amz-user-agent": "aws-sdk-js/3.523.0", "user-agent": "aws-sdk-js/3.523.0 ua/2.0 os/darwin#23.4.0 lang/js md/nodejs#20.10.0 api/s3#3.523.0", "amz-sdk-invocation-id": "7373b9d2-3439-45ac-a605-e4ec6ea28957", "amz-sdk-request": "attempt=1; max=3", "x-amz-date": "20240411T150044Z", "x-amz-content-sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "authorization": "AWS4-HMAC-SHA256 Credential=GK5bec374936673b0b236b1618/20240411/garage/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;host;if-match;x-amz-content-sha256;x-amz-date;x-amz-user-agent, Signature=868d4fca83d2e4672c671fc7bb2a270d98448bb7a198745941c1329df85a5c2d", "connection": "keep-alive"}, body: Body(Empty) } 2024-04-11 11:00:44 2024-04-11T15:00:44.778194Z DEBUG garage_api::s3::router: Received an unknown query parameter: 'x-id' 2024-04-11 11:00:44 2024-04-11T15:00:44.778198Z DEBUG garage_api::generic_server: Endpoint: GetObject 2024-04-11 11:00:44 2024-04-11T15:00:44.778856Z DEBUG garage_api::s3::get: Version meta: ObjectVersionMeta { headers: ObjectVersionHeaders { content_type: "text/plain", other: {} }, size: 6, etag: "a2ec0c77b7bea23455185bcc75535bf7" } 2024-04-11 11:00:44 2024-04-11T15:00:44.778929Z DEBUG garage_api::generic_server: 200 OK {"content-type": "text/plain", "last-modified": "Thu, 11 Apr 2024 15:00:30 GMT", "accept-ranges": "bytes", **"etag": "\"a2ec0c77b7bea23455185bcc75535bf7\""**, "content-length": "6"} ``` Note that the last line shows that the ETag of the existing object is "a2ec0c77b7bea23455185bcc75535bf7". The request sends the if-match header with value "6kjl35j6365k", so it doesn't match, and the request should return status 412 Precondition Failed By itself, If-Match on a GET request is rarely useful, but when making a Range request, it's essential that the range come from the same version of the object.