Difference between upload() and putObject() for uploading a file to S3?

Amazon Web-ServicesAmazon S3

Amazon Web-Services Problem Overview


In the aws-sdk's S3 class, what is the difference between upload() and putObject()? They seem to do the same thing. Why might I prefer one over the other?

Amazon Web-Services Solutions


Solution 1 - Amazon Web-Services

The advantage to using AWS SDK upload() over putObject() is as below:

  • If the reported MD5 upon upload completion does not match, it retries.
  • If the file size is large enough, it uses multipart upload to upload parts in parallel.
  • Retry based on the client's retry settings.
  • You can use for Progress reporting.
  • Sets the ContentType based on file extension if you do not provide it.

Solution 2 - Amazon Web-Services

upload() allows you to control how your object is uploaded. For example you can define concurrency and part size.

From their docs: Uploads an arbitrarily sized buffer, blob, or stream, using intelligent concurrent handling of parts if the payload is large enough.

One specific benefit I've discovered is that upload() will accept a stream without a content length defined whereas putObject() does not.

This was useful as I had an API endpoint that allowed users to upload a file. The framework delivered the file to my controller in the form of a readable stream without a content length. Instead of having to measure the file size, all I had to do was pass it straight through to the upload() call.

Solution 3 - Amazon Web-Services

When looking for the same information, I came across: https://aws.amazon.com/blogs/developer/uploading-files-to-amazon-s3/

This source is a little dated (referencing instead upload_file() and put() -- or maybe it is the Ruby SDK?), but it looks like the putObject() is intended for smaller objects than the upload().

It recommends upload() and specifies why:

> This is the recommended method of using the SDK to upload files to a > bucket. Using this approach has the following benefits: > > - Manages multipart uploads for objects larger than 15MB. > - Correctly opens files in binary mode to avoid encoding issues. > - Uses multiple threads for uploading parts of large objects in parallel.

Then covers the putObject() operation: > For smaller objects, you may choose to use #put instead.

EDIT: I was having problems with the .abort() operation on my .upload() and found this helpful: https://stackoverflow.com/questions/34429957/abort-stop-amazon-aws-s3-upload-aws-sdk-javascript

Now my various other events from https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Request.html are firing as well! With .upload() I only had 'httpUploadProgress'.

Solution 4 - Amazon Web-Services

This question was asked almost six years ago and I stumbled across it while searching for information on the latest AWS Node.js SDK (V3). While V2 of the SDK supports the "upload" and "putObject" functions, the V3 SDK only supports "Put Object" functionality as "PutObjectCommand". The ability to upload in parts is supported as "UploadPartCommand" and "UploadPartCopyCommand" but the standalone "upload" function available in V2 is not and there is no "UploadCommand" function.

So if you migrate to the V3 SDK, you will need to migrate to Put Object. Get Object is also different in V3. A Buffer is no longer returned and instead a readable stream or a Blob. So if you got the data through "Body.toString()" you now have to implement a stream reader or handle Blob's.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestioncallumView Question on Stackoverflow
Solution 1 - Amazon Web-Serviceserror2007sView Answer on Stackoverflow
Solution 2 - Amazon Web-Servicesrlay3View Answer on Stackoverflow
Solution 3 - Amazon Web-ServicesDougView Answer on Stackoverflow
Solution 4 - Amazon Web-ServicesRobert G. SchaffrathView Answer on Stackoverflow