Amazon S3 boto - how to create a folder?

Amazon S3Boto

Amazon S3 Problem Overview


How can I create a folder under a bucket using boto library for Amazon s3?

I followed the manual, and created the keys with permission, metadata etc, but no where in the boto's documentation it describes how to create folders under a bucket, or create a folder under folders in bucket.

Amazon S3 Solutions


Solution 1 - Amazon S3

There is no concept of folders or directories in S3. You can create file names like "abc/xys/uvw/123.jpg", which many S3 access tools like S3Fox show like a directory structure, but it's actually just a single file in a bucket.

Solution 2 - Amazon S3

Assume you wanna create folder abc/123/ in your bucket, it's a piece of cake with Boto

k = bucket.new_key('abc/123/')
k.set_contents_from_string('')

Or use the console

Solution 3 - Amazon S3

Use this:

import boto3
s3 = boto3.client('s3')
bucket_name = "YOUR-BUCKET-NAME"
directory_name = "DIRECTORY/THAT/YOU/WANT/TO/CREATE" #it's name of your folders
s3.put_object(Bucket=bucket_name, Key=(directory_name+'/'))

Solution 4 - Amazon S3

With AWS SDK .Net works perfectly, just add "/" at the end of the folder name string:

var folderKey =  folderName + "/"; //end the folder name with "/"
AmazonS3 client = Amazon.AWSClientFactory.CreateAmazonS3Client(AWSAccessKey, AWSSecretKey);
var request = new PutObjectRequest();
request.WithBucketName(AWSBucket);
request.WithKey(folderKey);
request.WithContentBody(string.Empty);
S3Response response = client.PutObject(request);

Then refresh your AWS console, and you will see the folder

Solution 5 - Amazon S3

Append "_$folder$" to your folder name and call put.

    String extension = "_$folder$";
    s3.putObject("MyBucket", "MyFolder"+ extension, new ByteArrayInputStream(new byte[0]), null);

see: http://www.snowgiraffe.com/tech/147/creating-folders-programmatically-with-amazon-s3s-api-putting-babies-in-buckets/

Solution 6 - Amazon S3

Tried many method above and adding forward slash / to the end of key name, to create directory didn't work for me:

client.put_object(Bucket="foo-bucket", Key="test-folder/")

You have to supply Body parameter in order to create directory:

client.put_object(Bucket='foo-bucket',Body='', Key='test-folder/')

Source: ryantuck in boto3 issue

Solution 7 - Amazon S3

It's really easy to create folders. Actually it's just creating keys.

You can see my below code i was creating a folder with utc_time as name.

Do remember ends the key with '/' like below, this indicates it's a key:

Key='folder1/' + utc_time + '/'

client = boto3.client('s3')
utc_timestamp = time.time()


def lambda_handler(event, context):

    UTC_FORMAT = '%Y%m%d'
    utc_time = datetime.datetime.utcfromtimestamp(utc_timestamp)
    utc_time = utc_time.strftime(UTC_FORMAT)
    print 'start to create folder for => ' + utc_time

    putResponse = client.put_object(Bucket='mybucketName',
                                    Key='folder1/' + utc_time + '/')

    print putResponse

Solution 8 - Amazon S3

Update for 2019, if you want to create a folder with path bucket_name/folder1/folder2 you can use this code:

from boto3 import client, resource

class S3Helper:

  def __init__(self):
      self.client = client("s3")
      self.s3 = resource('s3')

  def create_folder(self, path):
      path_arr = path.rstrip("/").split("/")
      if len(path_arr) == 1:
          return self.client.create_bucket(Bucket=path_arr[0])
      parent = path_arr[0]
      bucket = self.s3.Bucket(parent)
      status = bucket.put_object(Key="/".join(path_arr[1:]) + "/")
      return status

s3 = S3Helper()
s3.create_folder("bucket_name/folder1/folder2)

Solution 9 - Amazon S3

Although you can create a folder by appending "/" to your folder_name. Under the hood, S3 maintains flat structure unlike your regular NFS.

var params = {
Bucket : bucketName,
Key : folderName + "/"
};
s3.putObject(params, function (err, data) {});

Solution 10 - Amazon S3

S3 doesn't have a folder structure, But there is something called as keys.

We can create /2013/11/xyz.xls and will be shown as folder's in the console. But the storage part of S3 will take that as the file name.

Even when retrieving we observe that we can see files in particular folder (or keys) by using the ListObjects method and using the Prefix parameter.

Solution 11 - Amazon S3

Apparently you can now create folders in S3. I'm not sure since when, but I have a bucket in "Standard" zone and can choose Create Folder from Action dropdown.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
Questionvito huangView Question on Stackoverflow
Solution 1 - Amazon S3user240469View Answer on Stackoverflow
Solution 2 - Amazon S3TomNgView Answer on Stackoverflow
Solution 3 - Amazon S3Reza AmyaView Answer on Stackoverflow
Solution 4 - Amazon S3elranuView Answer on Stackoverflow
Solution 5 - Amazon S3Mamad AsgariView Answer on Stackoverflow
Solution 6 - Amazon S3azzamsaView Answer on Stackoverflow
Solution 7 - Amazon S3Gabriel WuView Answer on Stackoverflow
Solution 8 - Amazon S3Ilya VinnichenkoView Answer on Stackoverflow
Solution 9 - Amazon S3vpageView Answer on Stackoverflow
Solution 10 - Amazon S3BalaramView Answer on Stackoverflow
Solution 11 - Amazon S3PerelxView Answer on Stackoverflow