Object storage is one of the most cost-efficient technologies in the modern cloud stack, right up until it isn’t. Across finance, healthcare, and manufacturing, infrastructure teams routinely accumulate petabytes of unstructured data on object storage without a single lifecycle policy in place. What starts as a sensible architectural choice quietly becomes runaway spend, because the data grows and the access patterns shift, but the storage tier never does.
The 2025 Flexera State of the Cloud Report puts cloud waste at 27% of total spend, a figure that has held steady for three consecutive years. Storage is a disproportionate contributor: research from Komprise found that 55% of organisations spend 30% or more of their total budget on data storage alone, and the majority of that data is cold. It hasn’t been touched in months or years, yet it sits on a tier priced for millisecond retrieval.
This is not a procurement problem. It is an architecture problem, specifically the absence of a coherent lifecycle strategy. The good news is that the fix is mechanical: tiering rules, expiry policies, and a governance layer to keep them honest. Here is a concrete framework for implementing all three.
Understanding the tier model before you write a single policy
Every object storage system, whether S3-compatible, Azure Blob, or a managed European alternative, organises capacity into tiers differentiated by retrieval latency and cost. Getting the mental model right before writing rules matters, because a misconfigured tier boundary is as wasteful as no policy at all.
Hot is the tier for data that drives live operations: user sessions, current sensor readings, product catalogues. Latency is measured in single-digit milliseconds and the per-GB price reflects that. Warm covers data still actively queried but not in the critical path: recent transactional records, user profiles from the past quarter, last month’s logs. Cold is for historical analysis, compliance archives, and disaster recovery sources. Retrieval can tolerate minutes or hours, and the storage cost drops sharply. Archive is the floor: data that must be retained, possibly for regulatory reasons, but is almost never read. AWS S3 Glacier Deep Archive, for example, offers retrieval in hours at the lowest cost point in the S3 class hierarchy.
AWS S3 Intelligent-Tiering is worth understanding as a reference implementation. It moves objects to an infrequent-access tier after 30 days of no access, and to an archive instant-access tier after 90 days. Objects untouched for 180 days or more can drop to a deep archive tier, realising up to 95% in storage cost savings relative to the frequent-access baseline. One important caveat: objects smaller than 128 KB never move from the frequent-access tier, so Intelligent-Tiering’s economics do not apply to large volumes of small files. That caveat holds in spirit for any automated tiering system and must inform how you model your workloads.
The tier your data lands in at ingest is almost never the right tier six months later. Lifecycle policy is not an optimisation; it is the correction for a decision you made under uncertainty.
Building lifecycle rules that actually work in production
A lifecycle policy is a set of rules that evaluate objects against conditions and trigger transitions or deletions. The conditions are typically age (days since creation or last access), object prefix or tag, and size. The actions are tier transitions and expiry. Getting the rule structure right requires answering three questions before touching a console or writing a YAML file.
What is the access pattern of each object class? Log files, backup snapshots, medical imaging archives, and financial transaction records all have different decay curves. A web access log is nearly useless after 30 days for operational purposes but may need to be retained for 12 months for compliance. A DICOM imaging file may be accessed frequently for the first 90 days post-procedure, then rarely. Map the decay curve before you set the timer.
What is the retention floor? Regulatory obligations set a minimum, not a maximum. GDPR’s storage limitation principle requires that personal data is not kept longer than necessary for its purpose. Many teams interpret this as a reason to keep everything indefinitely; the correct interpretation is the opposite. Define the maximum retention period per data class and encode it as an expiry rule, not a mental note.
What are the retrieval cost implications? Moving data to a cold or archive tier reduces storage cost but introduces retrieval fees. If your incident response workflow pulls six months of logs every time an alert fires, archiving those logs will cost more in retrieval than you saved in storage. Model both sides before committing to a transition rule.
A practical rule structure for a finance or healthcare workload might look like this: raw ingest lands in hot; after 30 days without access, transition to warm (infrequent access); after 90 days, transition to cold; after the regulatory retention period, expire. Backup snapshots should carry an explicit expiry rule from the moment they are created, because unmanaged snapshots are one of the most consistent sources of compounding storage waste: 31% of Azure Blob Storage containers have not been accessed in over six months, and unmanaged snapshots account for roughly 11% of storage spend in a typical Azure environment.
S3-compatible storage tiers and the portability question
Most object storage systems in production today expose an S3-compatible API. That compatibility is valuable for portability but does not mean every provider’s tiering model maps cleanly onto S3’s storage class hierarchy. When you write lifecycle rules, you are writing them against a specific provider’s implementation, and the transitions available, the minimum storage durations, and the retrieval latency guarantees will differ.
We operate managed object storage on DIVERSITY’s infrastructure with S3-compatible APIs, which means lifecycle policies written against the S3 model translate directly, and data residency is guaranteed within European jurisdiction. For customers in regulated sectors, that combination matters: the storage tier your data lands in and the jurisdiction it sits in are both policy decisions, and they need to be made together.
If you are migrating from a hyperscaler, audit your existing S3 lifecycle configuration before lift-and-shift. Rules that reference storage classes by name (GLACIER, DEEP_ARCHIVE) may need to be remapped to the equivalent tier on the destination platform. The logic transfers; the class names may not.
Governance: the layer that keeps policies honest
Lifecycle rules decay. Buckets are created by new teams without inheriting organisational policies. Prefixes are renamed and fall outside existing rule scopes. Data classes that were warm last year are now cold, but no one updated the transition timer. Without a governance layer, a lifecycle policy written in Q1 is a liability by Q3.
The governance layer has three components. First, a tagging standard enforced at bucket creation: data class, owner, retention period, and environment. Without consistent tags, you cannot write rules that target the right objects, and you cannot report on policy coverage. Second, a policy audit on a defined cadence, at minimum quarterly, that checks every bucket for the presence of a lifecycle configuration and flags buckets where the last-modified date on the policy is older than six months. Third, cost allocation reporting broken down by storage tier, so that hot-tier spend on data older than 90 days is visible as a line item, not buried in a blended storage total.
Data retention automation is only as reliable as the process that maintains the rules that drive it. Treat lifecycle policies as infrastructure code, version them, review them in pull requests, and test transitions in a staging environment before applying them to production buckets.
Where to start when the backlog is already large
Most teams reading this have existing buckets with years of accumulated data and no lifecycle configuration. The instinct is to audit everything before touching anything. That is the wrong order of operations.
Start with the highest-cost buckets. Pull a storage cost report broken down by bucket, identify the top ten by spend, and apply a conservative policy to each: transition to cold after 90 days, no expiry yet. Run that for 30 days and measure the cost delta. That gives you a concrete number to present internally and builds confidence in the mechanism before you apply expiry rules to anything. Once the mechanism is trusted, extend coverage to the full bucket inventory and add expiry rules to data classes where the retention period is well-defined.
Object storage cost optimisation is not a one-time project. It is a practice: tiering rules set at ingest, expiry policies tied to retention requirements, and a governance process that keeps both honest as the data landscape changes. The teams that treat it as infrastructure rather than housekeeping are the ones whose storage costs scale sub-linearly with their data volume.
