Amazon S3 lifecycle retroactive application

Amazon Web-ServicesAmazon S3Amazon Glacier

Amazon Web-Services Problem Overview


Fairly straightforward question. Do Amazon S3 lifecycle rules that I set get applied to data retroactively? If so, what sort of delay might I see before older data begins to be archived or deleted?

By way of example, let's say I have a bucket with 3 years of backed up data. If I create a new lifecycle where that data will be archived after 31 days, and deleted after 365 days, will that new rule be applied to the existing data? How soon will it begin to be enforced?

Amazon Web-Services Solutions


Solution 1 - Amazon Web-Services

Yes it's retroactive (i.e. all the things that are already there and match will get the rule).
There may be a slight delay (i.e. rules will have day granularity and run on daily basis), but the rules will take effect immediately. Depending on how much data you have to remove/move it may take a while if you have a lot of pre-existing data.

Source: S3 FAQ here: http://aws.amazon.com/s3/faqs/

> Lifecycle policies apply to both existing and new S3 objects, ensuring > that you can optimize storage and maximize cost savings for all > current data and any new data placed in S3 without time-consuming > manual data review and migration. > > After an Object Expiration rule is added, the rule is applied to > objects that already exist in the bucket as well as new objects added > to the bucket. Once objects are past their expiration date, they are > identified and queued for removal. You will not be billed for storage > for objects on or after their expiration date, though you may still be > able to access those objects while they are in queue before they are > removed. As with standard delete requests, Amazon S3 doesn’t charge > you for removing objects using Object Expiration. You can set > Expiration rules for your versioning-enabled or versioning-suspended > buckets as well.

Solution 2 - Amazon Web-Services

S3 bucket lifecycle rules apply to both existing and new data in S3. In addiiton to Mircea answer:

  • Lifecycle rules run once a day at midnight Universal Coordinated Time (UTC). For example, if you create a lifecycle rule on October 15 at 4:00 PM Pacific Standard Time (PST) that transitions objects after one day, then the objects are transitioned to the new storage class on October 17 at 12:00 AM UTC.
  • New lifecycle rules can take up to 48 hours to complete the first run. If there are millions of objects for the rule, then it might take a few runs for all the objects to be transitioned into another storage class.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionDougWView Question on Stackoverflow
Solution 1 - Amazon Web-ServicesMirceaView Answer on Stackoverflow
Solution 2 - Amazon Web-ServicesIhor KonovalenkoView Answer on Stackoverflow