How can I easily determine if a Boto 3 S3 bucket resource exists?

PythonAmazon Web-ServicesAmazon S3Boto3

Python Problem Overview


For example, I have this code:

import boto3

s3 = boto3.resource('s3')

bucket = s3.Bucket('my-bucket-name')

# Does it exist???

Python Solutions


Solution 1 - Python

At the time of this writing there is no high-level way to quickly check whether a bucket exists and you have access to it, but you can make a low-level call to the HeadBucket operation. This is the most inexpensive way to do this check:

from botocore.client import ClientError

try:
    s3.meta.client.head_bucket(Bucket=bucket.name)
except ClientError:
    # The bucket does not exist or you have no access.

Alternatively, you can also call create_bucket repeatedly. The operation is idempotent, so it will either create or just return the existing bucket, which is useful if you are checking existence to know whether you should create the bucket:

bucket = s3.create_bucket(Bucket='my-bucket-name')

As always, be sure to check out the official documentation.

Note: Before the 0.0.7 release, meta was a Python dictionary.

Solution 2 - Python

As mentioned by @Daniel, the best way as suggested by Boto3 docs is to use head_bucket()

> head_bucket() - This operation is useful to determine if a > bucket exists and you have permission to access it.

If you have a small number of buckets, you can use the following:

>>> import boto3
>>> s3 = boto3.resource('s3')
>>> s3.Bucket('Hello') in s3.buckets.all()
False
>>> s3.Bucket('some-docs') in s3.buckets.all()
True
>>> 

Solution 3 - Python

I've had success with this:

import boto3

s3 = boto3.resource('s3')
bucket = s3.Bucket('my-bucket-name')

if bucket.creation_date:
   print("The bucket exists")
else:
   print("The bucket does not exist")

Solution 4 - Python

I tried Daniel's example and it was really helpful. Followed up the boto3 documentation and here is my clean test code. I have added a check for '403' error when buckets are private and return a 'Forbidden!' error.

import boto3, botocore
s3 = boto3.resource('s3')
bucket_name = 'some-private-bucket'
#bucket_name = 'bucket-to-check'

bucket = s3.Bucket(bucket_name)
def check_bucket(bucket):
    try:
        s3.meta.client.head_bucket(Bucket=bucket_name)
        print("Bucket Exists!")
        return True
    except botocore.exceptions.ClientError as e:
        # If a client error is thrown, then check that it was a 404 error.
        # If it was a 404 error, then the bucket does not exist.
        error_code = int(e.response['Error']['Code'])
        if error_code == 403:
            print("Private Bucket. Forbidden Access!")
            return True
        elif error_code == 404:
            print("Bucket Does Not Exist!")
            return False

check_bucket(bucket)

Hope this helps some new into boto3 like me.

Solution 5 - Python

Use lookup Function -> Returns None if bucket Exist

if s3.lookup(bucketName) is None:
    bucket=s3.create_bucket(bucketName) # Bucket Don't Exist
else:
    bucket = s3.get_bucket(bucketName) #Bucket Exist

Solution 6 - Python

you can use conn.get_bucket

from boto.s3.connection import S3Connection
from boto.exception import S3ResponseError    

conn = S3Connection(aws_access_key, aws_secret_key)

try:
    bucket = conn.get_bucket(unique_bucket_name, validate=True)
except S3ResponseError:
    bucket = conn.create_bucket(unique_bucket_name)

quoting the documentation at http://boto.readthedocs.org/en/latest/s3_tut.html

> As of Boto v2.25.0, this now performs a HEAD request (less expensive but worse error messages).

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionDanielView Question on Stackoverflow
Solution 1 - PythonDanielView Answer on Stackoverflow
Solution 2 - PythonhelloVView Answer on Stackoverflow
Solution 3 - Pythonhref_View Answer on Stackoverflow
Solution 4 - PythonmondnomView Answer on Stackoverflow
Solution 5 - PythonJacob OommenView Answer on Stackoverflow
Solution 6 - PythonvimView Answer on Stackoverflow