Object storage and the S3 storage classes

Where object storage fits, and where it does not

S3 stores objects in a flat namespace of buckets and keys, reached over an HTTP API, with capacity that is never provisioned. An object is written and replaced as a whole, which is the property that decides the classic wrong answer on this objective: a database updates a few bytes in the middle of a large file constantly, and object storage simply has no such operation. That mismatch is fundamental rather than a matter of configuration, and no amount of versioning, Availability Zone alignment or bucket policy changes it.

The corollary is the positive case. Anything written once and read many times — media files, backups, logs, static website content, the files a data lake is made of — fits object storage perfectly, and the effectively unlimited capacity is what makes it the default landing place for data whose eventual size nobody knows.

Choosing a class: two questions, in that order

How often is it read, and how long may it take to come back? Those two questions order the whole family. S3 Standard is for data read often and returned instantly, with no retrieval fee. Standard-IA and One Zone-IA are priced for data read perhaps monthly that must still come back instantly. Glacier Instant Retrieval sits below them for data touched about quarterly. Glacier Flexible Retrieval returns in minutes to hours, and Glacier Deep Archive — the cheapest storage of all — in hours.

Every class below Standard adds two things: a per-gigabyte retrieval charge and a minimum storage duration, which is thirty days for the infrequent-access classes, ninety for Glacier Flexible Retrieval and one hundred and eighty for Deep Archive. Deleting early is still billed for the remainder, which is why an archive class is wrong for data with a short life and irrelevant as an objection when a regulator has asked for seven years. And One Zone-IA is the one class that keeps its single copy in a single Availability Zone: right for data that is reproducible or is a second copy of something else, wrong for anything that exists nowhere else.

Intelligent-Tiering, and when it is the wrong answer

S3 Intelligent-Tiering watches each object's access pattern and moves it between access tiers itself, which makes it the answer to "nobody can say in advance which of these files will be read again". It charges a small monitoring and automation fee per object and no retrieval fee for its frequent and infrequent access tiers, so an object that turns out to be read daily is not punished for a rule somebody guessed.

Knowing when it is wrong is what makes it a choice rather than a default. Because the monitoring fee is per OBJECT, a bucket of many millions of very small files can spend more on monitoring than tiering saves. And when the pattern IS known — logs that are useless after ninety days — an explicit lifecycle rule is the better tool. The trade the exam draws is exactly that: automatic tiering for an unpredictable pattern, a lifecycle rule for a predictable one, and a fixed age is the wrong signal for data whose age says nothing about whether it is still being read.

What S3 does without being asked

Four defaults carry an unreasonable number of marks. Capacity is elastic and billed by what is stored, so a bucket is never sized. Buckets and objects are PRIVATE until somebody deliberately grants access, which is why an exposed bucket is always a customer configuration decision. Objects are stored redundantly across several Availability Zones in the bucket's Region — a bucket belongs to a Region and never to a zone — with One Zone-IA as the single exception. And a lifecycle rule changes an object's storage class in place, on a schedule you define, without any data leaving S3.

Versioning is the fifth thing worth switching on and the one a question about a bad script is fishing for: with versioning enabled an overwrite creates a new version and a delete adds a marker, so the previous content is still there to restore. Taken together these are the shared responsibility model in miniature — AWS keeps the object durable, you decide who may read it.

$ aws s3api head-object --bucket logs --key app.gz    "StorageClass": "STANDARD"$ aws s3 cp s3://logs/app.gz s3://logs/app.gz --storage-class GLACIER# nothing was downloaded and nothing left S3$ aws s3api head-object --bucket logs --key app.gz    "StorageClass": "GLACIER"