It&#39;s always `arn:PARTITION:s3:::NAME-OF-YOUR-BUCKET`. If you know the name of the bucket and in which partition it&#39;s located, you know the ARN. No need to &#39;get&#39; it from anywhere.

The PARTITION will be `aws`, `aws-us-gov`, or `aws-cn`depending on whether you&#39;re in general AWS, GovCloud, or China resepectively.

You can also select your S3 bucket ARN by selecting it using the tick mark at the s3 management console, which will pop up a Side bar. where there is a provision to copy your S3 bucket ARN.[S3 management console with bucket ARN][1] 


  [1]: https://i.stack.imgur.com/NP1ex.jpg

aws articles spell out the arn format but never say go here to see it. Highlighting my s3 bucket and seeing that **Copy Bucket ARN** kept me sane for a few more hours.

I&#39;m in the process of building an ASP.NET Core WebAPI and I&#39;m attempting to write unit tests for the controllers.  Most examples I&#39;ve found are from the older WebAPI/WebAPI2 platforms and don&#39;t seem to correlate with the new Core controllers.

My controller methods are returning `IActionResults`.  However, the `IActionResult` object only has a `ExecuteResultAsync()` method which requires a controller context.  I&#39;m instantiating the controller manually, so the controller context in this instance is null, which causes an exception when calling `ExecuteResultAsync`.  Essentially this is leading me down a very hacky path to get these unit tests to successfully complete and is very messy. I&#39;m left wondering that there must be a more simple/correct way of testing API controllers.

Also, my controllers are NOT using async/await if that makes a difference.

Simple example of what I&#39;m trying to achieve:

Controller method:

    [HttpGet(Name = &quot;GetOrdersRoute&quot;)]
    public IActionResult GetOrders([FromQuery]int page = 0)
    {
         try
         {
            var query = _repository.GetAll().ToList();
            
            int totalCount = query.Count;
            int totalPages = (int)Math.Ceiling((double)totalCount / pageSize) - 1;
            var orders = query.Skip(pageSize * page).Take(pageSize);

            return Ok(new
            {
               TotalCount = totalCount,
               TotalPages = totalPages,

               Orders = orders
            });
         }
         catch (Exception ex)
         {
            return BadRequest(ex);
         }
    }

Unit test:

	[Fact]
    public void GetOrders_WithOrdersInRepo_ReturnsOk()
    {
         // arrange
         var controller = new OrdersController(new MockRepository());

         // act
         IActionResult result = controller.GetOrders();
         
         // assert
         Assert.Equal(HttpStatusCode.OK, ????);
    }

Unit testing controller methods which return IActionResult

I have been launching Jupyter Notebook for years using the following command:

    jupyter-notebook --port=7000 --no-browser --no-mathjax
When I try to open the jupyter on the browser it ask me for a password, even though I have never set any before.
It is important to note that If I do set the port to a value different than `7000 (eg., the default 8888)` the interface will open with no problem

**I am running jupyter locally, and on the following setup:**

    Python 3.5.2

**With the following modules installed:**

    jupyter (1.0.0), jupyter-client (4.4.0), jupyter-console (5.0.0), jupyter-core (4.2.1), ipykernel (4.5.2), ipython (5.1.0), ipython-genutils (0.1.0), nbconvert (4.3.0), nbformat (4.2.0), notebook (4.3.0)

**NOTE: I have no jupyter configurations file**

Following are some of the output lines from the server:

    [I 19:16:24.358 NotebookApp] Serving notebooks from local directory: /Users/my_user_name
    [I 19:16:24.358 NotebookApp] 0 active kernels
    [I 19:16:24.358 NotebookApp] The Jupyter Notebook is running at: http://localhost:7000/?token=aa0dab6e2d85766f3e2e4f0f6633e4473db56a56c94cac76
    [I 19:16:24.358 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

And follwing are messages after I try to open it on the browser (using port 7000)

    [I 19:21:56.848 NotebookApp] 302 GET /tree (::1) 8.46ms
    [D 19:21:56.857 NotebookApp] Using contents: services/contents
    [D 19:21:56.919 NotebookApp] Path base/images/favicon.ico served from /usr/local/lib/python3.5/site-packages/notebook/static/base/images/favicon.ico
    [D 19:21:56.920 NotebookApp] Path components/jquery-ui/themes/smoothness/jquery-ui.min.css served from /usr/local/lib/python3.5/site-packages/notebook/static/components/jquery-ui/themes/smoothness/jquery-ui.min.css
    [D 19:21:56.922 NotebookApp] Path components/jquery-typeahead/dist/jquery.typeahead.min.css served from /usr/local/lib/python3.5/site-packages/notebook/static/components/jquery-typeahead/dist/jquery.typeahead.min.css
    [D 19:21:56.923 NotebookApp] Path style/style.min.css served from /usr/local/lib/python3.5/site-packages/notebook/static/style/style.min.css
    [D 19:21:56.925 NotebookApp] Path auth/css/override.css served from /usr/local/lib/python3.5/site-packages/notebook/static/auth/css/override.css
    [D 19:21:56.926 NotebookApp] Path components/es6-promise/promise.min.js served from /usr/local/lib/python3.5/site-packages/notebook/static/components/es6-promise/promise.min.js
    [D 19:21:56.926 NotebookApp] Path components/requirejs/require.js served from /usr/local/lib/python3.5/site-packages/notebook/static/components/requirejs/require.js
    [D 19:21:56.933 NotebookApp] Path base/images/logo.png served from /usr/local/lib/python3.5/site-packages/notebook/static/base/images/logo.png
    [D 19:21:56.934 NotebookApp] 200 GET /login?next=%2Ftree (::1) 80.86ms
    [D 19:21:57.001 NotebookApp] Path custom.css served from /usr/local/lib/python3.5/site-packages/notebook/static/custom/custom.css
    [D 19:21:57.003 NotebookApp] 304 GET /custom/custom.css (::1) 3.11ms
    [D 19:21:57.341 NotebookApp] Path auth/js/main.min.js served from /usr/local/lib/python3.5/site-packages/notebook/static/auth/js/main.min.js
    [D 19:21:57.344 NotebookApp] 200 GET /static/auth/js/main.min.js?v=20161219191623 (::1) 3.57ms

At this point there is a page from jupyter asking me to insert a password I have never set before.

## SOLVED! ##

With latest update of notebook module (4.3.1) the problem has been solved.

Once the jupyter notebook is launched the user is prompted to paste a URL into the browser at the first connection:

    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:7000/?token=32be0f5ee74cfe521187bc479855ce8b9fbab9e8021701c9

This solved the problem!

How to disable password request for a Jupyter notebook session?

Is it possible to get the ARN of an S3 bucket via the AWS command line?

I have looked through the documentation for `aws s3api ...` and `aws s3 ...` and have not found a way to do this. 

Get ARN of S3 Bucket with aws cli

<p>Is it possible to get the ARN of an S3 bucket via the AWS command line?</p>
<p>I have looked through the documentation for <code>aws s3api ...</code> and <code>aws s3 ...</code> and have not found a way to do this.</p>


I am getting error &quot;This policy contains the following error: The policy must contain a valid version string For more information about the IAM policy grammar&quot;  even i included version in my policy when trying to create a new policy in AWS. My policy is

    {
      &quot;Version&quot;: &quot;2015-06-19&quot;,
      &quot;Statement&quot;: [
        {
          &quot;Effect&quot;: &quot;Allow&quot;,
          &quot;Action&quot;: &quot;s3:*&quot;,
          &quot;Resource&quot;: [
            &quot;arn:aws:s3:::repo.com&quot;,
            &quot;arn:aws:s3:::repo.com/*&quot;
          ]
        }
      ]
    }

AWS Policy must contain valid version string

I am getting an error when calling to `assume role` method of STS. It says that the user is not authorized to perform `sts:AsumeRole` on resource `xxx`.

I did the following:

 1. I created a role to access to S3 bucket.
 2. I ran a test over policy simulator and works fine
 3. I created a new group, and in it, i created a new policy that
    enables all sts actions, over all resources.
 4. I ran a test with the policy simulator, to sts assume role, pointing
    to the ARN of role created at step one; and it works fine
 5. I created a new user, and put it in group created at step 3
 6. With the credentials of the new user, i try to get a new credentials
    using sts asume role, but throw me an error that say my user is not
    authorized to perform sts:AssumeRole

What am I doing wrong?

Policy in Group

    {
        &quot;Version&quot;: &quot;2012-10-17&quot;,
        &quot;Statement&quot;: [
            {
                &quot;Sid&quot;: &quot;some-large-id&quot;,
                &quot;Effect&quot;: &quot;Allow&quot;,
                &quot;Action&quot;: [
                    &quot;sts:*&quot;
                ],
                &quot;Resource&quot;: [
                    &quot;*&quot;
                ]
            }
        ]
    }

Policy in role

    {
        &quot;Version&quot;: &quot;2012-10-17&quot;,
        &quot;Statement&quot;: [
            {
                &quot;Sid&quot;: &quot;another-large-id&quot;,
                &quot;Effect&quot;: &quot;Allow&quot;,
                &quot;Action&quot;: [
                    &quot;s3:PutObject&quot;
                ],
                &quot;Resource&quot;: [
                    &quot;arn:aws:s3:::my-bucket-name/*&quot;
                ]
            }
        ]
    }

And finally calling like this

    let policy = {
    	&quot;Version&quot;: &quot;2012-10-17&quot;,
    	&quot;Statement&quot;: [
    		{
    			&quot;Sid&quot;: &quot;new-custom-id&quot;,
    			&quot;Effect&quot;: &quot;Allow&quot;,
    			&quot;Action&quot;: [&quot;s3:PutObject&quot;],
    			&quot;Resource&quot;: [&quot;arn:aws:s3:::my-bucket-name/*&quot;]
    		}
    	]
    };
    
    let params = {
    	DurationSeconds: 3600, 
    	ExternalId: &#39;some-value&#39;, 
    	Policy: JSON.stringify(policy), 
    	RoleArn: &quot;arn:aws:iam::NUMBER:role/ROLE-NAME&quot;, //Cheked, role is the same that step one
    	RoleSessionName: this.makeNewSessionId()
    };
    let sts = new AWS.STS({ apiVersion: &#39;2012-08-10&#39; });
    
    sts.assumeRole(params, (err, data) =&gt; {
    	if(err) console.log(err);
    	else console.log(data);
    });

How enable access to AWS STS AssumeRole

I&#39;m trying to push a docker image to the AWS ECR repository using the aws-cli.

 - I just run the `get-login` command
 - execute the output (which returns `login succeeded`) 
 - then try to push a docker image then I get the
   message:  
`denied: Your Authorization Token has expired. Please run
   &#39;aws ecr get-login&#39; to fetch a new one.`

I don&#39;t know whats going wrong,  I&#39;m pushing to the right repo, the time on my mac is correct. 
This was working before, but since I reinstalled my mac and upgraded to macOS Sierra it&#39;s not working anymore, so probably related to that.

My `aws --version` output:

`aws-cli/1.11.34 Python/2.7.10 Darwin/16.3.0 botocore/1.4.91`

The complete output of the commands I run:

    $ aws ecr get-login --region eu-west-1
    docker login -u AWS -p AQECAHh....b6Wk -e none https://1234567890.dkr.ecr.eu-west-1.amazonaws.com
    $ docker login -u AWS -p AQECAHh....b6Wk -e none https://1234567890.dkr.ecr.eu-west-1.amazonaws.com
    Flag --email has been deprecated, will be removed in 1.13.
    Login Succeeded
    $ docker push 1234567890.dkr.ecr.eu-west-1.amazonaws.com/service-web:latest
    The push refers to a repository [1234567890.dkr.ecr.eu-west-1.amazonaws.com/service-web]
    c1f87971dfa9: Preparing 
    2eb644aea3de: Preparing 
    9c8843ffe48e: Preparing 
    39bb58d049d4: Preparing 
    f053bc969599: Preparing 
    7169084246b8: Waiting 
    bb134a1936fd: Waiting 
    184e76848a1c: Waiting 
    75c8fcf65748: Waiting 
    eb9b9ee1ea58: Waiting 
    f4bf35723edd: Waiting 
    ddffe1a64b3c: Waiting 
    fd1a1154db16: Waiting 
    b542e946067a: Waiting 
    d49ed2a5e1ed: Waiting 
    bb39b980367a: Waiting 
    25b8358d062f: Waiting 
    997eee521fc7: Waiting 
    50b5447183a8: Waiting 
    4339b5cb0e1d: Waiting 
    3dbd4a53b21b: Waiting 
    2bec16216500: Waiting 
    b9fd8e264df6: Waiting 
    b6ca02dfe5e6: Waiting 
    denied: Your Authorization Token has expired. Please run &#39;aws ecr get-login&#39; to fetch a new one.



`Authorization Token has expired` issue AWS-CLI on MacOS Sierra

I am using a JSON structure to store details of a person. Basically name, phone, registeredTimestamp etc., will be other attributes, and Primary key will be the email address.

markedLocations, visitedLocations, searchHistory and recommendation are the lists or Map as AWS calls it. It will contain many JSON objects.

JSON Structure-

    {
    	&quot;name&quot;:&quot;Mama Miya&quot;,
    	&quot;phone&quot;:&quot;9383883223&quot;,
    	&quot;registeredTimestamp&quot;:&quot;some-time&quot;,
    	&quot;lastActiveTimestamp&quot;:&quot;some-time&quot;,
    	&quot;signinType&quot;:&quot;facebook&quot;,
    	&quot;additionalSigninData&quot;:{
    		&quot;key1&quot;:&quot;value1&quot;,
    		&quot;key2&quot;:&quot;value2&quot;
    	},
    	&quot;markedLocations&quot;:[
    		{
    			&quot;latitude&quot;:&quot;77.33&quot;,
    			&quot;longitude&quot;:&quot;12.33&quot;,
    			&quot;name&quot;:&quot;Home&quot;,
    			&quot;description&quot;:&quot;my new home&quot;,
    			&quot;placeid&quot;:&quot;55334433&quot;,
    			&quot;image&quot;:&quot;url/abc.png&quot;
    		}
    	]
    }



From the app, if the user visits a new location, I need to add a new JSON Object to the *markedLocations*.
So the markedLocations should look something like-

    &quot;markedLocations&quot;:[
		{
			&quot;latitude&quot;:&quot;77.33&quot;,
			&quot;longitude&quot;:&quot;12.33&quot;,
			&quot;name&quot;:&quot;Home&quot;,
			&quot;description&quot;:&quot;my new home&quot;,
			&quot;placeid&quot;:&quot;55334433&quot;,
			&quot;image&quot;:&quot;url/abc.png&quot;
		},{
			&quot;latitude&quot;:&quot;22.11&quot;,
			&quot;longitude&quot;:&quot;22.55&quot;,
			&quot;name&quot;:&quot;Ocean&quot;,
			&quot;description&quot;:&quot;Ocean&quot;,
			&quot;placeid&quot;:&quot;32423423&quot;,
			&quot;image&quot;:&quot;url/icean.png&quot;
		}
	]

Code/Schema-

    var item = {
            &#39;email&#39;: {&#39;S&#39;: req.body.email},
            &#39;phone&#39;: {&#39;S&#39;: req.body.phone},
            &#39;registeredTimestamp&#39;: {&#39;S&#39;: req.body.registeredTimestamp},
            &#39;lastActiveTimestamp&#39;: {&#39;S&#39;: req.body.lastActiveTimestamp},
            &#39;signinType&#39;: {&#39;S&#39;: req.body.signinType},
            &#39;version&#39;: {&#39;S&#39;: req.body.version},
            &#39;markedLocations&#39;: {&#39;L&#39;: req.body.markedLocations}
        };

I checked -

- In DynamoDB how do I append an element to a list field using Java [link][1]

- updating a JSON array in AWS dynamoDB [link][2]

- Updating a Set in Dynamo db using Node Js [link][3]

- how to update item in dynamoDB using nodejs? [link][4]

- And checked AWS docs for [UpdateItem API][5]. I didn&#39;t find anything relevant to my problem/doubt.


Please can you tell me if there is a way to append the JSON array with a new JSON object in NodeJS? I am not able proceed on how to implement the same.

**OR**

Should I create separate table for each Array and have email as primary key and timestamp as sort key and proceed?


PS: I am new to DynamoDB, I&#39;ve worked with MongoDB earlier and there are ways to work with JSON arrays and objects there. Unable to find a way here.

  [1]: https://stackoverflow.com/questions/41177319/in-dynamodb-how-do-i-append-an-element-to-a-list-field-using-java
  [2]: https://stackoverflow.com/questions/40317443/updating-a-json-array-in-aws-dynamodb
  [3]: https://stackoverflow.com/questions/33708603/updating-a-set-in-dynamo-db-using-node-js
  [4]: https://stackoverflow.com/questions/33858475/how-to-update-item-in-dynamodb-using-nodejs
  [5]: http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_UpdateItem.html

Append a new object to a JSON Array in DynamoDB using NodeJS

When I run CloudFormation `deploy` using a template with API Gateway resources, the first time I run it, it creates and deploys to stages. The subsequent times I run it, it updates the resources but doesn&#39;t deploy to stages.

Is that behaviour as intended?  If yes, how&#39;d I get it to deploy to stages whenever it updates?

(Terraform mentions a similar issue: https://github.com/hashicorp/terraform/issues/6613)


CloudFormation doesn&#39;t deploy to API gateway stages on update

After all the tough work of migration etc, I just realised that I need to serve the content using CNAME (e.g media.abc.com). The bucket name needs to start with media.abc.com/S3/amazon.com to ensure it works perfectly. 

I just realised that S3 doesn&#39;t allow direct rename from the console. 

Is there any way to work around this?

How to rename AWS S3 Bucket

When I log to my **S3 console** I am unable to download multiple selected files (the WebUI allows downloads only when one file is selected):

https://console.aws.amazon.com/s3

Is this something that can be changed in the user policy or is it a limitation of Amazon?


Amazon S3 console: download multiple files at once

Let&#39;s say that I have a machine that I want to be able to write to a certain log file stored on an S3 bucket.

So, the machine needs to have writing abilities to that bucket, but, I don&#39;t want it to have the ability to overwrite or delete any files in that bucket (including the one I want it to write to). 

So basically, I want my machine to be able to only append data to that log file, without overriding it or downloading it.

Is there a way to configure my S3 to work like that? Maybe there&#39;s some IAM policy I can attach to it so it will work like I want?

Append data to an S3 object

I am using the below to get the [stack information][1] I want via AWS Cli:

    aws cloudformation --region ap-southeast-2 describe-stacks --stack-name mystack

It&#39;s returning result OK:

    {
        &quot;Stacks&quot;: [
            {
                &quot;StackId&quot;: &quot;arn:aws:mystackid&quot;, 
                &quot;LastUpdatedTime&quot;: &quot;2017-01-13T04:59:17.472Z&quot;, 
                &quot;Tags&quot;: [], 
                &quot;Outputs&quot;: [
                    {
                        &quot;OutputKey&quot;: &quot;Ec2Sg&quot;, 
                        &quot;OutputValue&quot;: &quot;sg-97e13dff&quot;
                    }, 
                    {
                        &quot;OutputKey&quot;: &quot;DbUrl&quot;, 
                        &quot;OutputValue&quot;: &quot;myUrl&quot;
                    }
                ], 
                &quot;CreationTime&quot;: &quot;2017-01-13T03:27:18.893Z&quot;, 
                &quot;StackName&quot;: &quot;mystack&quot;, 
                &quot;NotificationARNs&quot;: [], 
                &quot;StackStatus&quot;: &quot;UPDATE_ROLLBACK_COMPLETE&quot;, 
                &quot;DisableRollback&quot;: false
            }
        ]
    }

But I do not know how to return only the value of OutputValue which is myUrl

As I do not need the rest, just myUrl.

Is that possible via aws cloudformation describe-stacks?

**Edit**

I just realize I can use --[query][2]:

    --query &quot;Stacks[0].Outputs[1].OutputValue&quot;

will get exactly what I want but I would like to use DbUrl else if the number of Outputs changes, my result will be unexpected.


  [1]: http://docs.aws.amazon.com/cli/latest/reference/cloudformation/describe-stacks.html
  [2]: http://docs.aws.amazon.com/cli/latest/userguide/controlling-output.html

Getting Outputs from aws cloudformation describe-stacks

I have an S3 bucket that is 100% empty.  Versioning was never enabled on the bucket.  However, I still cannot remove the bucket.  I have tried via the Console and the CLI tool.  On the console it just says &quot;Error&quot; with no error message.  From the cli and api it tells me: &quot;An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: The bucket you tried to delete is not empty&quot;.  I have tried all of the following:

`aws s3 rb s3://&lt;bucket_name&gt; --force` -&gt; BucketNotEmpty

`aws s3 rm s3://&lt;bucket_name&gt; --recursive` -&gt; No output (because it&#39;s already empty)

`aws s3api list-object-versions --bucket &lt;bucket_name&gt;` -&gt; No output (because versioning was never enabled)

`aws s3api list-multipart-uploads --bucket &lt;bucket_name&gt;` -&gt; No outputs

`aws s3api list-objects --delimiter=/ --prefix= --bucket &lt;bucket_name&gt;` -&gt; No Output (because it&#39;s empty)

* It has no dependencies (it&#39;s not used by cloudfront or anything else that I&#39;m aware of).
* The bucket has been empty for approximately 5 days.
* I was able to delete another very similar bucket with the same IAM user.  Additionally my IAM user has Admin access.

Can&#39;t Delete Empty S3 Bucket

For AWS CLI configuration and credentials files how do you comment out lines in these files - I checked the documentation here http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#cli-config-files but found nothing.

e.g.

~/.aws/credentials

    [default]
    aws_access_key_id = XXXXXXXXX
    aws_secret_access_key = YYYYYYYY
    
    [qa]
    aws_access_key_id = AAAAAAAAA
    aws_secret_access_key = BBBBBB


~/.aws/config

    [default]
    region = us-east-1
    
    [profile qa]
    region = us-west-2
    
    [profile staging]
    region = us-east-2



How do you comment out lines in AWS CLI config and credentials files?

I tried to follow this [tutorial][1]. 

This is what I did in the console:

    pip3 install --user --upgrade awscli

after that, when I write: 

    pip3 --version

I&#39;m getting:

    pip 9.0.1 from /Users/user/Library/Python/3.4/lib/python/site-packages (python 3.4)

then I wrote:

    pip3 install --user --upgrade awscli

this command downloaded `awscli` and automatically added this:

    # Setting PATH for Python 3.4
    # The orginal version is saved in .profile.pysave
    PATH=&quot;/Library/Frameworks/Python.framework/Versions/3.4/bin:${PATH}&quot;
    export PATH

to my `.profile`

Later on, just to be sure, I wrote:

    source ~/.profile

and then when I type:

    user$ aws
    -bash: aws: command not found

I restarted the terminal with no luck also. 

What&#39;s the problem here?


  [1]: http://docs.aws.amazon.com/cli/latest/userguide/cli-install-macos.html

Content Type	Original Author	Original Content on Stackoverflow
Question	ljcundiff	View Question on Stackoverflow
Solution 1 - Amazon Web-Services	Sergey Kovalev	View Answer on Stackoverflow
Solution 2 - Amazon Web-Services	jovel	View Answer on Stackoverflow
Solution 3 - Amazon Web-Services	deawar	View Answer on Stackoverflow

Get ARN of S3 Bucket with aws cli

Amazon Web-Services Problem Overview

Amazon Web-Services Solutions

Solution 1 - Amazon Web-Services

Solution 2 - Amazon Web-Services

Solution 3 - Amazon Web-Services

How to disable password request for a Jupyter notebook session?

Unit testing controller methods which return IActionResult

Attributions