CloudFront + S3 Website: "The specified key does not exist" when an implicit index document should be displayed

I've just deployed a static website to Amazon S3, which can currently be viewed here: http://www.rdegges.com.s3-website-us-east-1.amazonaws.com/

If you click any of the article links, you'll notice the following error:

S3 is complaining the file doesn't exist. Now, here's what's weird about this -- I'm using CloudFront on my domain. So when you click that article link, it's sending the request to CloudFront which then tries to fetch the file back from the S3 bucket.

However, if you visit that same URL from S3 directly, eg: http://www.rdegges.com.s3-website-us-east-1.amazonaws.com/2015/building-a-heroku-addon-planning/ the page will load just fine.

It appears that something is getting lost in translation here.

Anyone got a suggestion of what I can do to fix my settings?

I'll go out on a limb and say that the specified key doesn't technically exist, so the error message is technically accurate but doesn't tell the whole story. This should be an easy fix.

S3 buckets have two¹ endpoints, "REST" and "website." They have two different feature sets. The web site endpoint provides magical resolution of index documents (e.g. index.html, which appears to be what is actually supposed to be returned to the browser in the example you provided) while the REST endpoints don't.

When you configure CloudFront in front of a bucket used for web site hosting, you usually don't want to configure the origin as an "S3" origin by selecting the bucket name from the drop-down list; instead, you want to configure it as a "Custom" origin, and use the web site endpoint hostname as provided in the S3 console (e.g. example-bucket.s3-website-us-east-1...) because otherwise, CloudFront assumes you want it to use the REST endpoint for the bucket (which allows authentication and private content, which the web site endpoint doesn't).

>Important

>Do not select the name of your bucket from the list, for example, example.com.s3.amazonaws.com.

>http://docs.aws.amazon.com/gettingstarted/latest/swh/getting-started-create-cfdist.html</strike>

The documentation was refactored since this question was originally answered, so the message shown above now appears one page later, and has been reworded, but the gist is the same. The "name of the bucket" seems to refer to the choices shown in the drop-down, which is not what you want.

>Note

>Be sure to specify the static website hosting endpoint, not the name of the bucket.

>http://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-cloudfront-walkthrough.html

The hint that you're using the REST endpoint for the bucket is that the error message is returned in XML. If you were using the web site endpoint, then the web site endpoint would return error messages in HTML.

Create a new origin for the CloudFront distribution, as described, then change the behavior to send requests to the new origin, then send a CloudFront cache invalidation request for /* and you should be set.

See also:

http://docs.aws.amazon.com/AmazonS3/latest/dev/WebsiteEndpoints.html#WebsiteRestEndpointDiff

---

¹ two endpoints. Technically, there are more than two, since all buckets have at least two possible REST endpoint hostnames... but there are two types of endpoints. Buckets also have an optional transfer acceleration endpoint that uses the AWS edge network (the same infrastructure that powers CloudFront) for faster/optimized transfers, particularly from geographic locations more distant from the region where the bucket is provisioned, but without using the CloudFront cache. This endpoint looks like https://example-bucket.s3-accelerate.amazonaws.com if you activate it, and carries an additional usage charge for most requests since you are using more of the AWS network and less of the public Internet... but, that is a difference in the behind-the-scenes deployment of the endpoint, not the behavior of the endpoint. The transfer acceleration endpoint is still a REST endpoint, so just like the other REST endpoints, it does not have the web site hosting features. CloudFront won't let you use an acceleration endpoint for an origin domain name, because that wouldn't make sense -- if such a configuration were allowed, requests and responses would loop through the AWS Edge Network twice and increase both latency and costs without providing any benefit.

Encountered same issue and how how I resolved it was in CloudFront Origin Settings set Origin Domain Name to <website bucket>.s3-website-us-west-2.amazonaws.com

In CloudFront Generate Settings make sure to have index.html as Default Root Object.

In S3 make sure to have Use this bucket to host a website selected and set index.html as Index document.

Are you hosting a React Site on S3? Possibly with React Router?

NOTE: I have my S3 Bucket closed away from public access, and I did not have any desire to make it public. Also, while the problem I faced is the same as what is asked above, the solution in my case is different.

In case you, like me, do NOT want to make your S3 endpoint public, and are facing this issue with React Router where direct URLs with routes are returning a 403 Forbidden on CloudFront, read on:

This is happening because CloudFront expects the S3 bucket to have the file that your URL supposedly points to.

For instance, if you have a static site "example.com" and a route "/stack", CloudFront will try to look for the 'stack' file in your S3 bucket. This obviously does not exist, and CloudFront will return an error.

Best way to resolve this is to set up a custom Error in CloudFront for HTTP error code 403. The custom error response can point to the "index.html" page with a "200 OK" status. This will solve the error, point the user to the right file, and at the same time, retain the original URL (example.com/stack) which will then be picked up by React Router.

Access Cloudfront -> Your Distribution

Goto Error Pages and Click Add Error Page . Enter details like the below

See AWS docs on how to use CloudFront to serve a static website hosted on S3 over https.

Transcribing content below for convenience (or in case the link ever goes bad).

---

1. Use the Amazon S3 console to create a bucket and to enable static website hosting on the bucket.

2. From the Static website hosting dialog box, copy the Endpoint of your bucket without the leading http://. The format is similar to bucketname.s3-website-region.amazonaws.com. You need the endpoint in this format for a later step.

3. Add a bucket policy that allows public read access to the bucket that you created.

4. Create a CloudFront web distribution. Be sure to configure the following:

• For Origin Domain Name, enter the endpoint that you copied in step 2.
• For Allowed HTTP Methods, select GET, HEAD, OPTIONS.
• For Alternate Domain Names (CNAMEs), enter the CNAME you want to use for your website.

5. If you don't want to use SSL (HTTPS) for your website, proceed to the next step. If you want to use SSL for your website, you can choose Request or Import a Certificate with ACM to request a certificate. For more information, see Using Alternate Domain Names and HTTPS.

6. Choose Create Distribution.

7. Update the DNS records for your domain to point your website's CNAME to your CloudFront distribution's domain name. You can find your distribution's domain name in the CloudFront console in a format that is similar to d1234abcd.cloudfront.net.

8. Wait for your DNS changes to propagate and for the previous DNS entries to expire.

I was also having a similar problem I have followed these steps which have solved this issue

STEPS:

->go to CloudFront Distributions 
->click the ID
->after Clicking Id you will find different categories like General, Origins and Origin Groups .
->Click the Origins and Origin Groups
->Click the checkbox  of your s3 bucket and click edit
->under grand read Permissions on Bucket click "Yes, Update bucket policy"

This step has solved my issue.

I was facing the same issue, how I solved it, below are the steps

• Go to Amazon S3 and then go to your bucket directory

• Enable S3 Static Website Hosting

• Go to Permissions tab -> Block all public access -> uncheck Block all public access and save it

• Go to Permissions tab -> Bucket Policy -> Edit and Save

  {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicRead",
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:GetObject",
                "s3:GetObjectVersion"
            ],
            "Resource": "arn:aws:s3:::your-bucket-name/*"
        }
    ]
  }
  

Note: change this line "Resource": "arn:aws:s3:::your-bucket-name/*"

Ex: "Resource": "arn:aws:s3:::ng-ci-cd-aws-demo/*"

reference: https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html

There are a few requirements to get s3 websites working with Cloudfront:

• Objects in the bucket must be publicly accessible.
• Objects in the bucket can't be encrypted by AWS Key Management Service (AWS KMS).
• The bucket policy must allow access to s3:GetObject.
• If the bucket policy grants public read access, then the AWS account that owns the bucket must also own the object.
• The requested objects must exist in the bucket.
• Amazon S3 Block Public Access must be disabled on the bucket.
• If Requester Pays is enabled, then the request must include the request-payer parameter.
• If you're using a Referer header to restrict access from CloudFront to your S3 origin, then review the custom header.

https://aws.amazon.com/premiumsupport/knowledge-center/s3-website-cloudfront-error-403/

The main issue I had was, there are two types of s3 buckets urls:

REST endpoints format: https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#access-bucket-intro

https://bucket-name.s3.Region.amazonaws.com

Website endpoints format: https://docs.aws.amazon.com/AmazonS3/latest/dev/WebsiteEndpoints.html

http://bucket-name.s3-website-Region.amazonaws.com
http://bucket-name.s3-website.Region.amazonaws.com

In my case I needed to use the website url not the REST url:

• REST: http://project-name.s3.us-east-1.amazonaws.com
• Website: http://project-name.s3-website-us-east-1.amazonaws.com

Using the website url will allow you to use the Website hosting features of S3 such as the index/error documents and redirect rules!

You should follow the steps below.

To update your bucket policy using the CloudFront console, follow these steps:

 1. Open the CloudFront console and choose your distribution.
 2. Choose the Origins and Origin Groups tab.
 3. Select the S3 origin and choose Edit.
 4. For Restrict Bucket Access, choose Yes.
 5. For Origin Access Identity, choose the existing identity or create a new one.
 6. For Grant Read Permissions on Bucket, choose Yes, Update Bucket Policy.
 7. Choose Yes, Edit.

If you configured an OAI, the OAI must be included in the S3 bucket policy If you added an OAI to your CloudFront distribution, you must also include an allow statement for the OAI in the S3 bucket policy.

To verify if your bucket policy allows the OAI, open your S3 bucket in the Amazon S3 console. Then, choose the Permissions tab and review the bucket policy. The following is an example allow statement for an OAI:

{
  "Sid": "1",
  "Effect": "Allow",
  "Principal": {
    "AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity EAF5XXXXXXXXX"
  },
  "Action": "s3:GetObject",
  "Resource": "arn:aws:s3:::DOC-EXAMPLE-BUCKET/*"
}

Reference: https://aws.amazon.com/tr/premiumsupport/knowledge-center/s3-rest-api-cloudfront-error-403/

I had this problem when I was trying to include the bucket name in my route53 redirect for a customer, to make it easier e.g:

https://0832234.signin.aws.amazon.com/console/s3/?bucket=clientbucket.com

Clicking on "All buckets" and going back into the clients bucket / or removing the bucket from the url did the trick, I can now download and open the files.