Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CompleteMultipartUpload root XML element bug fix #413

Closed
wants to merge 1 commit into from

Conversation

amosov-f
Copy link

Bug with root XML element
Actual: MultipartUpload
Expected: CompleteMultipartUpload
https://docs.aws.amazon.com/AmazonS3/latest/API/API_CompleteMultipartUpload.html

Bug with root XML element
Actual: MultipartUpload
Expected: CompleteMultipartUpload
https://docs.aws.amazon.com/AmazonS3/latest/API/API_CompleteMultipartUpload.html
@PeterMcKinnis
Copy link

I was just about to submit the exact same PR.

Here are some details

Describe the bug

The S3 function CompletedMultipartUpload doesn't work because the generated payload xml looks like this

<MultipartUpload>
             <Part>
                <PartNumber>1</PartNumber>
               <ETag>"a54357aff0632cce46d942af68356b38"</ETag>
             </Part>
             ....
</MultipartUpload>

instead of

<CompleteMultipartUpload>
             <Part>
                <PartNumber>1</PartNumber>
               <ETag>"a54357aff0632cce46d942af68356b38"</ETag>
             </Part>
             ....
</CompleteMultipartUpload>

This is caused by a bug here which is easily fixed.

// aws_client\lib\src\generated\s3\v2006_03_01.dart - line 335

      /// 'MultipartUpload' in the following line should be 'CompleteMultipartUpload'
      payload: multipartUpload?.toXml('MultipartUpload'),

However, looks like this code is auto generated, so maybe it's a bug with the API definition or the code generator? Not sure.

Expected Behavior

Calls to 'CompleteMultipartUpload' work.

Current Behavior

Calls to 'CompleteMultipartUpload' fail with a MalformedXML exception.

Reproduction Steps

You will need to enter your own S3 credentials and a path to any file larger than 8Mb on your system.

import 'dart:io';
import 'package:aws_s3_api/s3-2006-03-01.dart' as s3;

void main() async {
  final client = s3.S3(
      region: "My Region",
      endpointUrl: "My S3 EndpointUrl",
      credentials: s3.AwsClientCredentials(
          accessKey: "My AccessKey", secretKey: "My SecretKey"));

  await uploadMultiPart(client, bucket: "My Bucket", key: "TempKeyAHKIQ", file: File(r"My Path To Any File larger than 8Mb"));

}

Future<void> uploadMultiPart(s3.S3 client,
    {required String bucket,
    required String key,
    required File file,
    Map<String, String>? metadata,
    int maxChunkSize = 8 * 1 << 20}) async {
  // Start Multi Part Upload
  var upload = await client.createMultipartUpload(
      bucket: bucket, key: key, metadata: metadata);
  var uploadId = upload.uploadId!;

  // Upload Chunks
  var fileReader = await file.open();
  var partNumber = 1;
  var parts = <s3.CompletedPart>[];
  while (true) {
    var bytes = await fileReader.read(maxChunkSize);
    if (bytes.isEmpty) {
      break;
    }
    var part = await client.uploadPart(
        bucket: bucket,
        key: key,
        partNumber: partNumber,
        uploadId: uploadId,
        body: bytes);
    var eTag = part.eTag!;
    parts.add(s3.CompletedPart(eTag: eTag, partNumber: partNumber));
    partNumber += 1;
  }

  // Complete Multi-Part Upload
  await client.completeMultipartUpload(
      bucket: bucket,
      key: key,
      multipartUpload: s3.CompletedMultipartUpload(parts: parts),
      uploadId: uploadId);
}

Possible Solution

No response

Additional Information/Context

See official documentation here

SDK version used

Dart 3.4.0-99.0.dev (dev)

Environment details (OS name and version, etc.)

"Windows 10 Home" 10.0 (Build 19045)

@Schwusch
Copy link
Collaborator

@dreznichuk seems to have solved this one in #419

@Schwusch Schwusch closed this Mar 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants