Downloading folders from aws s3, cp or sync?

If I want to download all the contents of a directory on S3 to my local PC, which command should I use cp or sync ?

Any help would be highly appreciated.

For example,

if I want to download all the contents of "this folder" to my desktop, would it look like this ?

 aws s3 sync s3://"myBucket"/"this folder" C:\\Users\Desktop

Using aws s3 cp from the AWS Command-Line Interface (CLI) will require the --recursive parameter to copy multiple files.

aws s3 cp s3://myBucket/dir localdir --recursive

The aws s3 sync command will, by default, copy a whole directory. It will only copy new/modified files.

aws s3 sync s3://mybucket/dir localdir

Just experiment to get the result you want.

Documentation:

• cp command

• sync command

Just used version 2 of the AWS CLI. For the s3 option, there is also a --dryrun option now to show you what will happen:

aws s3 --dryrun cp s3://bucket/filename /path/to/dest/folder --recursive

In the case you want to download a single file, you can try the following command:

aws s3 cp s3://bucket/filename /path/to/dest/folder

In case you need to use another profile, especially cross account. you need to add the profile in the config file

[profile profileName]
region = us-east-1
role_arn = arn:aws:iam::XXX:role/XXXX
source_profile = default

and then if you are accessing only a single file

aws s3 cp s3://crossAccountBucket/dir localdir --profile profileName

---

You've many options to do that, but the best one is using the AWS CLI.

Here's a walk-through:

1. Download and install AWS CLI in your machine:

   • Install the AWS CLI using the MSI Installer (Windows).

   • Install the AWS CLI using the Bundled Installer for Linux, OS X, or Unix.

2. Configure AWS CLI:

Make sure you input valid access and secret keys, which you received when you created the account.

3. Sync the S3 bucket using:

    aws s3 sync s3://yourbucket/yourfolder /local/path
   

In the above command, replace the following fields:

• yourbucket/yourfolder >> your S3 bucket and the folder that you want to download.
• /local/path >> path in your local system where you want to download all the files.

sync method first lists both source and destination paths and copies only differences (name, size etc.).

cp --recursive method lists source path and copies (overwrites) all to the destination path.

If you have possible matches in the destination path, I would suggest sync as one LIST request on the destination path will save you many unnecessary PUT requests - meaning cheaper and possibly faster.