[s3cmd](http://s3tools.org/s3cmd) and [AWS CLI](https://aws.amazon.com/cli/) are both command line tools.  They&#39;re well suited if you want to script your deployment through shell scripting (e.g. bash).

AWS CLI gives you simple file-copying abilities through the &quot;s3&quot; command, which should be enough to deploy a static website to an S3 bucket. It also has some small advantages such as being pre-installed on Amazon Linux, if that was where you were working from (it&#39;s also [easily installable through `pip`](http://docs.aws.amazon.com/cli/latest/userguide/installing.html#install-with-pip)).

One AWS CLI command that may be appropriate to sync a local directory to an S3 bucket:

    $ aws s3 sync . s3://mybucket

Full documentation on this command:
http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html

_Edit: As mentioned by @simon-buchan in a comment, the `aws s3api` command gives you access to the complete S3 API, but its interface is more &quot;raw&quot;._

s3cmd supports everything AWS CLI does, plus adds some more extended functionality on top, although I&#39;m not sure you would require any of it for your purposes.  You can see all its commands here:
http://s3tools.org/usage

Installation of s3cmd may be a bit more involved because it doesn&#39;t seem to be packages for it in any distros main repos.

[boto](http://docs.pythonboto.org/en/latest/) is a Python library, and in fact the official AWS Python SDK.  The AWS CLI, also being written in Python, actually uses part of the boto library ([botocore](http://botocore.readthedocs.org/en/latest/)).  It would be well suited only if you were writing your deployment scripts in Python.  There are official SDKs for other popular languages (Java, PHP, etc.) should you prefer:
http://aws.amazon.com/tools/

The rawest form of access to S3 is through AWS&#39;s REST API.  Everything else is built upon it at some point.  If you feel adventurous, here&#39;s the S3 REST API documentation:
http://docs.aws.amazon.com/AmazonS3/latest/API/APIRest.html

I&#39;m using C++ (not C++11). I need to make a pointer to a function inside a class. I try to do following:

    void MyClass::buttonClickedEvent( int buttonId ) {
    	// I need to have an access to all members of MyClass&#39;s class
    }
    
    void MyClass::setEvent() {
    
    	void ( *func ) ( int );	
    	func = buttonClickedEvent; // &lt;-- Reference to non static member function must be called
    	
    }
    
    setEvent();

But there&#39;s an error: &quot;Reference to non static member function must be called&quot;. What should I do to make a pointer to a member of MyClass?

Reference to non-static member function must be called

I used git for the first time and I set my user name and user mail. The commands I used are below:

    git config --global user.email &quot;bob@example.com&quot;
    git config user.email &quot;bob@example.com&quot;
    git config --global user.name &quot;bob&quot;
    git config user.name &quot;bob&quot;

When I run `git commit --author &quot;bob&quot;` , I got an error `fatal: No existing author found with &#39;bob&#39;`. How can I set user name and email?

fatal: No existing author found with &#39;XXX&#39;

I am thinking about redeploying my static website to Amazon S3. I need to automate the deployment so I was looking for an API for such tasks. I&#39;m a bit confused over the different options.

**Question**: What is the difference between s3cmd, the Python library boto and AWS CLI?

Difference between s3cmd, boto and AWS CLI

I am thinking about redeploying my static website to Amazon S3. I need to automate the deployment so I was looking for an API for such tasks. I'm a bit confused over the different options.
Question: What is the difference between s3cmd, the Python library boto and AWS CLI?

I get an error `AWS::S3::Errors::InvalidRequest The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.` when I try upload file to S3 bucket in new Frankfurt region. All works properly with `US Standard` region.

Script:

    backup_file = &#39;/media/db-backup_for_dev/2014-10-23_02-00-07/slave_dump.sql.gz&#39;
    s3 = AWS::S3.new(
        access_key_id:     AMAZONS3[&#39;access_key_id&#39;],
        secret_access_key: AMAZONS3[&#39;secret_access_key&#39;]
    )
 
    s3_bucket = s3.buckets[&#39;test-frankfurt&#39;]
 
    # Folder and file name
    s3_name = &quot;database-backups-last20days/#{File.basename(File.dirname(backup_file))}_#{File.basename(backup_file)}&quot;
 
    file_obj = s3_bucket.objects[s3_name]
    file_obj.write(file: backup_file)

aws-sdk (1.56.0)

How to fix it?

Thank you.

The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256

I have an Amazon S3 bucket that I make public with a policy like this

    {
    	&quot;Version&quot;: &quot;2012-10-17&quot;,
    	&quot;Statement&quot;: [
    		{
    			&quot;Sid&quot;: &quot;Allow Public Access to All Objects&quot;,
    			&quot;Effect&quot;: &quot;Allow&quot;,
    			&quot;Principal&quot;: &quot;*&quot;,
    			&quot;Action&quot;: &quot;s3:GetObject&quot;,
    			&quot;Resource&quot;: &quot;arn:aws:s3:::bucket/*&quot;
    		}
    	]
    }

My bucket is now visible as
http://bucket.s3-website-us-east-1.amazonaws.com/

I see others refer to their bucket as
http://s3-us-east-1.amazonaws.com/bucket/

I would prefer the 2nd URL, but it gives an Access Denied.

How can I change my policy to allow the 2nd URL ?


URL for public Amazon S3 bucket

I&#39;ve recently inherited a Rails app that uses S3 for storage of assets. I have transferred all assets to my S3 bucket with no issues. However, when I alter the app to point to the new bucket I get 403 Forbidden Status. 

My S3 bucket is set up with the following settings:

**Permissions**

Everyone can list

**Bucket Policy**

    {
	 &quot;Version&quot;: &quot;2012-10-17&quot;,
	 &quot;Statement&quot;: [
		{
			&quot;Sid&quot;: &quot;PublicReadGetObject&quot;,
			&quot;Effect&quot;: &quot;Allow&quot;,
			&quot;Principal&quot;: &quot;*&quot;,
			&quot;Action&quot;: &quot;s3:GetObject&quot;,
			&quot;Resource&quot;: &quot;arn:aws:s3:::bucketname/*&quot;
		}
	 ]
    }

**CORS Configuration**

    &lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;
    &lt;CORSConfiguration xmlns=&quot;http://s3.amazonaws.com/doc/2006-03-01/&quot;&gt;
        &lt;CORSRule&gt;
            &lt;AllowedOrigin&gt;*&lt;/AllowedOrigin&gt;
            &lt;AllowedMethod&gt;GET&lt;/AllowedMethod&gt;
            &lt;MaxAgeSeconds&gt;3000&lt;/MaxAgeSeconds&gt;
        &lt;/CORSRule&gt;
        &lt;CORSRule&gt;
            &lt;AllowedOrigin&gt;https://www.appdomain.com&lt;/AllowedOrigin&gt;
            &lt;AllowedMethod&gt;PUT&lt;/AllowedMethod&gt;
            &lt;AllowedMethod&gt;POST&lt;/AllowedMethod&gt;
            &lt;AllowedMethod&gt;DELETE&lt;/AllowedMethod&gt;
            &lt;AllowedHeader&gt;*&lt;/AllowedHeader&gt;
        &lt;/CORSRule&gt;
    &lt;/CORSConfiguration&gt;

**Static Web Hosting**

Enabled.

What else can I do to allow the public to reach these assets?

Amazon S3 bucket returning 403 Forbidden

I need to check if a file exists using AWS SDK. Here is what I&#39;m doing:

	var params = {
		Bucket: config.get(&#39;s3bucket&#39;),
		Key: path
	};

	s3.getSignedUrl(&#39;getObject&#39;, params, callback);

It works but the problem is that when the object doesn&#39;t exists, the callback (with arguments err and url) returns no error, and when I try to access the URL, it says &quot;NoSuchObject&quot;.

Shouldn&#39;t this `getSignedUrl` method return an error object when the object doesn&#39;t exists? How do I determine if the object exists? Do I really need to make a call on the returned URL?


How to determine if object exists AWS S3 Node.JS sdk

For example, I have this code:

    import boto3

    s3 = boto3.resource(&#39;s3&#39;)

    bucket = s3.Bucket(&#39;my-bucket-name&#39;)

    # Does it exist???

How can I easily determine if a Boto 3 S3 bucket resource exists?

I have S3 access only to a specific directory in an S3 bucket.

For example, with the `s3cmd` command if I try to list the whole bucket:

        $ s3cmd ls s3://bucket-name

I get an error: `Access to bucket &#39;my-bucket-url&#39; was denied`

But if I try access a specific directory in the bucket, I can see the contents:

        $ s3cmd ls s3://bucket-name/dir-in-bucket

Now I want to connect to the S3 bucket with python boto. Similary with:

        bucket = conn.get_bucket(&#39;bucket-name&#39;)

I get an error: `boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden`

But if I try:

        bucket = conn.get_bucket(&#39;bucket-name/dir-in-bucket&#39;)

The script stalls for about 10 seconds, and prints out an error afterwards. Bellow is the full trace. Any idea how to proceed with this?

Note question is about the boto version 2 module, not boto3.

	Traceback (most recent call last):
	  File &quot;test_s3.py&quot;, line 7, in &lt;module&gt;
		bucket = conn.get_bucket(&#39;bucket-name/dir-name&#39;)
	  File &quot;/usr/local/lib/python2.7/dist-packages/boto/s3/connection.py&quot;, line 471, in get_bucket
		return self.head_bucket(bucket_name, headers=headers)
	  File &quot;/usr/local/lib/python2.7/dist-packages/boto/s3/connection.py&quot;, line 490, in head_bucket
		response = self.make_request(&#39;HEAD&#39;, bucket_name, headers=headers)
	  File &quot;/usr/local/lib/python2.7/dist-packages/boto/s3/connection.py&quot;, line 633, in make_request
		retry_handler=retry_handler
	  File &quot;/usr/local/lib/python2.7/dist-packages/boto/connection.py&quot;, line 1046, in make_request
		retry_handler=retry_handler)
	  File &quot;/usr/local/lib/python2.7/dist-packages/boto/connection.py&quot;, line 922, in _mexe
		request.body, request.headers)
	  File &quot;/usr/lib/python2.7/httplib.py&quot;, line 958, in request
		self._send_request(method, url, body, headers)
	  File &quot;/usr/lib/python2.7/httplib.py&quot;, line 992, in _send_request
		self.endheaders(body)
	  File &quot;/usr/lib/python2.7/httplib.py&quot;, line 954, in endheaders
		self._send_output(message_body)
	  File &quot;/usr/lib/python2.7/httplib.py&quot;, line 814, in _send_output
		self.send(msg)
	  File &quot;/usr/lib/python2.7/httplib.py&quot;, line 776, in send
		self.connect()
	  File &quot;/usr/lib/python2.7/httplib.py&quot;, line 1157, in connect
		self.timeout, self.source_address)
	  File &quot;/usr/lib/python2.7/socket.py&quot;, line 553, in create_connection
		for res in getaddrinfo(host, port, 0, SOCK_STREAM):
	socket.gaierror: [Errno -2] Name or service not known

Python boto, list contents of specific dir in bucket

I have the next code:

    users_table = Table(users_table_name, connection=Core.aws_dynamodb_connection)
    users_table.put_item(data={
      &quot;login&quot;: login,
      &quot;password&quot;: hashlib.sha256(password.encode(&quot;utf-8&quot;)).hexdigest(),
      &quot;profile&quot;: profile,
      &quot;registration_date&quot;: datetime.now() # PROBLEM IS HERE
    })

But when I run it, it fails with error:

&gt; TypeError: Unsupported type &quot;&lt; type &#39;datetime.datetime&#39; &gt;&quot; for value &quot;2015-01-12 05:02:57.053131&quot;

I&#39;ve tried a lot of ways, but it seems that it isn&#39;t possible to save `datetime` to DynamoDB. Btw it works fine in MongoDB.

Is there any solution?

Is it possible to save datetime to DynamoDB?

I&#39;m trying to do a &quot;hello world&quot; with new [boto3][1] client for AWS.

The use-case I have is fairly simple: get object from S3 and save it to the file.

In boto 2.X I would do it like this:

    import boto
    key = boto.connect_s3().get_bucket(&#39;foo&#39;).get_key(&#39;foo&#39;)
    key.get_contents_to_filename(&#39;/tmp/foo&#39;)

In boto 3 . I can&#39;t find a clean way to do the same thing, so I&#39;m manually iterating over the &quot;Streaming&quot; object:

    import boto3
    key = boto3.resource(&#39;s3&#39;).Object(&#39;fooo&#39;, &#39;docker/my-image.tar.gz&#39;).get()
    with open(&#39;/tmp/my-image.tar.gz&#39;, &#39;w&#39;) as f:
        chunk = key[&#39;Body&#39;].read(1024*8)
        while chunk:
            f.write(chunk)
            chunk = key[&#39;Body&#39;].read(1024*8)

or

    import boto3
    key = boto3.resource(&#39;s3&#39;).Object(&#39;fooo&#39;, &#39;docker/my-image.tar.gz&#39;).get()
    with open(&#39;/tmp/my-image.tar.gz&#39;, &#39;w&#39;) as f:
        for chunk in iter(lambda: key[&#39;Body&#39;].read(4096), b&#39;&#39;):
            f.write(chunk)


And it works fine. I was wondering is there any &quot;native&quot; boto3 function that will do the same task?
 

  [1]: https://github.com/boto/boto3

How to save S3 object to a file using boto3

How can I see what&#39;s inside a bucket in S3 with `boto3`? (i.e. do an `&quot;ls&quot;`)?

Doing the following:

    import boto3
    s3 = boto3.resource(&#39;s3&#39;)
    my_bucket = s3.Bucket(&#39;some/path/&#39;)

returns:

    s3.Bucket(name=&#39;some/path/&#39;)

How do I see its contents?

Listing contents of a bucket with boto3

I need to create a CSV and upload it to an S3 bucket. Since I&#39;m creating the file on the fly, it would be better if I could write it directly to S3 bucket as it is being created rather than writing the whole file locally, and then uploading the file at the end.

Is there a way to do this? My project is in Python and I&#39;m fairly new to the language. Here is what I tried so far:

    import csv
    import csv
    import io
    import boto
    from boto.s3.key import Key


    conn = boto.connect_s3()
    bucket = conn.get_bucket(&#39;dev-vs&#39;)
    k = Key(bucket)
    k.key = &#39;foo/foobar&#39;

    fieldnames = [&#39;first_name&#39;, &#39;last_name&#39;]
    writer = csv.DictWriter(io.StringIO(), fieldnames=fieldnames)
    k.set_contents_from_stream(writer.writeheader())

I received this error: BotoClientError: s3 does not support chunked transfer

**UPDATE: I found a way to write directly to S3, but I can&#39;t find a way to clear the buffer without actually deleting the lines I already wrote. So, for example:**

    conn = boto.connect_s3()
    bucket = conn.get_bucket(&#39;dev-vs&#39;)
    k = Key(bucket)
    k.key = &#39;foo/foobar&#39;

    testDict = [{
        &quot;fieldA&quot;: &quot;8&quot;,
        &quot;fieldB&quot;: None,
        &quot;fieldC&quot;: &quot;888888888888&quot;},
        {
        &quot;fieldA&quot;: &quot;9&quot;,
        &quot;fieldB&quot;: None,
        &quot;fieldC&quot;: &quot;99999999999&quot;}]

    f = io.StringIO()
    fieldnames = [&#39;fieldA&#39;, &#39;fieldB&#39;, &#39;fieldC&#39;]
    writer = csv.DictWriter(f, fieldnames=fieldnames)
    writer.writeheader()
    k.set_contents_from_string(f.getvalue())

    for row in testDict:
        writer.writerow(row)
        k.set_contents_from_string(f.getvalue())

    f.close()

Writes 3 lines to the file, however I&#39;m unable to release memory to write a big file. If I add:

    f.seek(0)
    f.truncate(0)
to the loop, then only the last line of the file is written. Is there any way to release resources without deleting lines from the file?

Can you upload to S3 using a stream rather than a local file?

I have tried both `s3cmd`:

    $ s3cmd -r -f -v del s3://my-versioned-bucket/

And the AWS CLI:

    $ aws s3 rm s3://my-versioned-bucket/ --recursive

But both of these commands simply add `DELETE` markers to S3. The command for removing a bucket also doesn&#39;t work (from the AWS CLI):

    $ aws s3 rb s3://my-versioned-bucket/ --force
    Cleaning up. Please wait...
    Completed 1 part(s) with ... file(s) remaining
    remove_bucket failed: s3://my-versioned-bucket/ A client error (BucketNotEmpty) occurred when calling the DeleteBucket operation: The bucket you tried to delete is not empty. You must delete all versions in the bucket.

Ok... how? There&#39;s no information in [their documentation][1] for this. S3Cmd says it&#39;s a &#39;fully-featured&#39; S3 command-line tool, but it makes [no reference to versions][2] other than its own. Is there any way to do this without using the web interface, which will take forever and requires me to keep my laptop on?


  [1]: http://docs.aws.amazon.com/cli/latest/reference/s3/index.html
  [2]: http://s3tools.org/usage

How do I delete a versioned bucket in AWS S3 using the CLI?

How to exclude multiple folders while using aws s3 syn ?

I tried :

```
    # aws s3 sync s3://inksedge-app-file-storage-bucket-prod-env \ 
                  s3://inksedge-app-file-storage-bucket-test-env \
                  --exclude &#39;reportTemplate/* orders/* customers/*&#39;
```

But still it&#39;s doing sync for folder &quot;customer&quot;

Output :

```
    copy: s3://inksedge-app-file-storage-bucket-prod-env/customers/116/miniimages/IMG_4800.jpg
       to s3://inksedge-app-file-storage-bucket-test-env/customers/116/miniimages/IMG_4800.jpg

    copy: s3://inksedge-app-file-storage-bucket-prod-env/customers/116/miniimages/DSC_0358.JPG
       to s3://inksedge-app-file-storage-bucket-test-env/customers/116/miniimages/DSC_0358.JPG
```


Exclude multiple folders using AWS S3 sync

I am following the [AWS CLI Setup guide][1]. I have managed to successfully install the tool on my Mac OS X terminal with the following output:

    Running cmd: /usr/bin/python virtualenv.py --python /usr/bin/python /Users/fr/.local/lib/aws
    Running cmd: /Users/fr/.local/lib/aws/bin/pip install --no-index --find-links file:///Users/fr/Downloads/awscli-bundle/packages awscli-1.5.3.tar.gz
    You can now run: /Users/fr/.local/lib/aws/bin/aws --version

My issue is that I have to type the full path **/Users/fr/.local/lib/aws/bin/aws** to execute any **aws** command. As per the guide&#39;s final step, I should be able to execute aws command directly without typing the absolute path for it to execute.

When I try using just **aws**, I get the following output:

    aws --version                                                                                         ⏎
    command not found: aws

I followed the instructions to make sure that **~/bin** is in my **PATH** environment and I could see there was no output, hence, I executed the export **PATH=~/bin:$PATH** command to add **~/bin to $PATH**. But this has made no difference. The **aws** command does not work directly.

Could someone please advise what could be going wrong?

  [1]: http://docs.aws.amazon.com/cli/latest/userguide/installing.html#install-bundle-other-os

AWS CLI $PATH Settings

I&#39;m starting a bash script which will take a path in S3 (as specified to the [ls][1] command) and dump the contents of all of the file objects to `stdout`. Essentially I&#39;d like to replicate `cat /path/to/files/*` except for S3, e.g. `s3cat &#39;/bucket/path/to/files/*&#39;`. My first inclination looking at the options is to use the `cp` command to a temporary file and then `cat` that.

Has anyone tried this or similar or is there already a command I&#39;m not finding which does it?

  [1]: http://docs.aws.amazon.com/cli/latest/reference/s3/ls.html

How to use AWS S3 CLI to dump files to stdout in BASH?

I am going to create a new distribution at **CloudFront**. Already I have uploaded my SSL certificate at AWS IAM using AWS CLI. That certificate appears in the Custom SSL Certificate dropdown on new distribution page but it is **DISABLED**.

Can someone tell me why is it so ? How to select my custom SSL certificate for this distribution ?

Unable to select Custom SSL Certificate (stored in AWS IAM)

I am trying to use aws container service as per the documentation in http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_GetStarted.html

The below error is thrown when running the command: 
     
    aws ecs list-container-instances --cluster default

    You must specify a region. You can also configure your region by running &quot;aws configure&quot;.

The documentation does not mention anything about specifying a default region. How do we do it in a console?


Content Type	Original Author	Original Content on Stackoverflow
Question	user1781186	View Question on Stackoverflow
Solution 1 - Amazon S3	mfisherca	View Answer on Stackoverflow

Difference between s3cmd, boto and AWS CLI

Amazon S3 Problem Overview

Amazon S3 Solutions

Solution 1 - Amazon S3

fatal: No existing author found with 'XXX'

Reference to non-static member function must be called

Attributions