S3 – Lifecycle Management

  • Is to manage the lifecycle of objects in the bucket
  • Lifecycle configuration enables you to simplify the lifecycle management of the objects, such as automated transition of less-frequently accessed objects to low-cost storage and schedule deletions.
  • 1000 lifecycle rules can be configured per bucket
  • Examples:
    • If you are uploading periodic logs in your bucket, your application might need those logs for a week or a month after creation, and after that you might want to delete them.
    • Some documents are frequently accessed for a certain period of time. After that they are less frequently accessed. Overtime, you might not need real time access to those objects, but organization requires them to archive for a longer period and optionally delete them after
    • You might also upload some types of data into S3 primarily for archival purposes, for example digital media archives, financial and healthcare records, raw genomics sequence data, long-term database backups and data that must be retained for regulatory compliance
  • Lifecycle configuration can be specified as a XML. This comprises a set of rules with predefined actions that you want Amazon S3 to perform on objects during their lifetime which includes:
    • Transition actions in which you define objects to transition into a different storage class. For eg:, you may choose to transition objects to S3-IA 0 days after creation, to Glacier an year after creation
    • Expiration actions in which you specify when the object expires. Then amazon s3 deletes the expired objects.
  • Amazon S3 stores the Lifecycle configuration as a “lifecycle” subresource attached to your bucket.
  • Using Amazon S3 API, you can PUT, DELETE or GET a Lifecycle configuration.
  • Can be configured using Amazon S3 management console or programmatically by using the AWS SDK wrapper libraries, and also can make REST API calls directly.
  • Transitioning Objects:  Following transitions are supported.
    • From the STANDARD or REDUCED REDUNDANCY to STANDARD_IA.
      • But S3 does not transition objects less than 128KB’s in size to the S3-IA class. It is not effective for smaller objects and S3 will not transition them
      • Objects must be stored for 30 days in current storage class before they are moved to S3-IA. For eg, you cannot create a lifecycle rule to transition an object into S3-IA a day after it’s creation. It’s not supported because often younger objects accessed more frequently or deleted sooner than is suitable for S3-IA
    • From any storage class to Glacier
    • Combine these rules manage an object’s complete lifecycle, including a first transition to S3-IA, a 2nd transition to Glacier and an expiration
    • The following are not supported transitions
      • S3-IA to S3or S3-RRS
      • GLACIER to any other storage class
      • From any storage class to S3-RRS
    • Transitioning to GLACIER:
      • Objects in the GLACIER storage class are not available in real time.

Archived objects are Amazon S3 objects, but before you can access an archived object, you must first restore a temporary copy of it. The restored object copy is available only for the duration you specify in the restore request. After that, Amazon S3 deletes the temporary copy, and the object remains archived in Amazon Glacier.

Note that object restoration from an archive can take up to five hours.

You can restore an object by using the Amazon S3 console or programmatically by using the AWS SDKs wrapper libraries or the Amazon S3 REST API in your code.

  • Transition of objects to GLACIER is only one way.

If you want to change the storage class of an already archived object to either Standard or RRS, you must use the restore operation to make a temporary copy first. Then use the copy operation to overwrite the object as a STANDARD, STANDARD_IA, or REDUCED_REDUNDANCY object.

  • The GLACIER storage class objects are visible and available only through Amazon S3, not through Amazon Glacier.
    Amazon S3 stores the archived objects in Amazon Glacier; however, these are Amazon S3 objects, and you can access them only by using the Amazon S3 console or the API. You cannot access the archived objects through the Amazon Glacier console or the API.
  • Expiring Objects:
    • When an object reaches the end of its lifetime, Amazon S3 queues it for removal and removes it asynchronously. There may be a delay between the expiration date and the date at which Amazon S3 removes an object. You are not charged for an expired object.
  • Lifecycle and other Bucket Configurations:
    • Lifecycle can be enabled on both nonversioned buckets and versioning enabled buckets.
    • Versioning enabled buckets maintain one current and zero or more noncurrent object versions. You can define separate lifecycle rules for current and noncurrent versions
    • Lifecycle configuration on MFA enabled buckets is not supported.

Leave a comment