Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(middleware-flexible-checksums): perform checksum calculation and validation by default #6750

Merged

Conversation

trivikr
Copy link
Member

@trivikr trivikr commented Dec 19, 2024

Issue

Description

Performs checksum calculation and validation by default based on values in RequestChecksumCalculation and ResponseChecksumValidation

Testing

Unit testing in CI

Verified that CRC32 checksums are computed by default

import { S3 } from "../aws-sdk-js-v3/clients/client-s3/dist-cjs/index.js";
import { NodeHttpHandler } from "../aws-sdk-js-v3/node_modules/@smithy/node-http-handler/dist-cjs/index.js";
import { equal } from "assert";

// Prints checksum headers for request and response.
class CustomHandler extends NodeHttpHandler {
  constructor() {
    super();
  }

  printChecksumHeaders(prefix, headers) {
    for (const [header, value] of Object.entries(headers)) {
      if (
        header.startsWith("x-amz-checksum-") ||
        header.startsWith("x-amz-sdk-checksum-")
      ) {
        console.log(`${prefix}['${header}']: '${value}'`);
      }
    }
  }

  async handle(request, options) {
    this.printChecksumHeaders("request", request.headers);
    const response = await super.handle(request, options);
    this.printChecksumHeaders("response", response.response.headers);
    return response;
  }
}

// WHEN_SUPPORTED is default.
const client = new S3({
  requestHandler: new CustomHandler(),
});
const Bucket = "test-flexible-checksums-v2"; // Replace with your test bucket name.
const Key = "hello-world.txt";
const Body = "Hello World"; // Replace with the content you want to test.

console.log("Put Object");
// ChecksumAlgorithm is not explicitly required to be specified.
await client.putObject({ Bucket, Key, Body });

console.log("\nGet Object");
// ChecksumMode is not explicitly required to be specified.
const response = await client.getObject({ Bucket, Key });

equal(Body, await response.Body.transformToString());

Output

Put Object
request['x-amz-sdk-checksum-algorithm']: 'CRC32'
request['x-amz-checksum-crc32']: 'ShexVg=='
response['x-amz-checksum-crc32']: 'ShexVg=='
response['x-amz-checksum-type']: 'FULL_OBJECT'

Get Object
request['x-amz-checksum-mode']: 'ENABLED'
response['x-amz-checksum-crc32']: 'ShexVg=='
response['x-amz-checksum-type']: 'FULL_OBJECT'

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Copy link
Contributor

@kuhe kuhe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add more tests in middleware-flexible-checksums.integ.spec.ts?

@trivikr trivikr force-pushed the flexchecks-default-checksum-calculation-validation branch from b49d70e to 8ad8496 Compare December 20, 2024 02:30
@trivikr
Copy link
Member Author

trivikr commented Dec 20, 2024

Add more tests in middleware-flexible-checksums.integ.spec.ts?

The tests were added in clients/client-s3/test/unit/flexibleChecksums.spec.ts in prior diff.
However, the integration tests need to as close to the source code as possible - so generic tests were moved to middleware package in #6752

And the tests were added to updated file in d6af53c (#6750)

@trivikr trivikr force-pushed the flexchecks-default-checksum-calculation-validation branch 2 times, most recently from 249c196 to c57d066 Compare January 10, 2025 17:48
@trivikr trivikr force-pushed the flexchecks-default-checksum-calculation-validation branch from c57d066 to c450f57 Compare January 14, 2025 21:43
@trivikr trivikr marked this pull request as ready for review January 14, 2025 21:43
@trivikr trivikr requested a review from a team as a code owner January 14, 2025 21:43
@trivikr trivikr merged commit f6068c8 into aws:main Jan 15, 2025
4 checks passed
@trivikr trivikr deleted the flexchecks-default-checksum-calculation-validation branch January 15, 2025 00:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants