The Problem
Your application generates more than 10 TB of logs per month. Compliance requires retaining logs for 10 years. The team regularly accesses logs from the past month but rarely touches anything older. Storing 10 years of 10+ TB/month data in S3 Standard would cost a fortune.
The Solution
Store logs in Amazon S3 and use lifecycle policies to move logs older than 1 month to S3 Glacier Deep Archive.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
aws s3api put-bucket-lifecycle-configuration \
--bucket my-application-logs \
--lifecycle-configuration '{
"Rules": [
{
"ID": "ArchiveOldLogs",
"Status": "Enabled",
"Filter": {
"Prefix": "logs/"
},
"Transitions": [
{
"Days": 30,
"StorageClass": "DEEP_ARCHIVE"
}
],
"Expiration": {
"Days": 3650
}
}
]
}'
This single rule handles the entire 10-year journey: hot storage for 30 days, cheapest archival for the rest, and automatic deletion at year 10.
How It Works
The Cost Math
At 10 TB/month over 10 years, you accumulate roughly 1,200 TB of total data:
| Strategy | Approximate Monthly Cost at Scale |
|---|---|
| All in S3 Standard | ~$27,600/month (1,200 TB) |
| S3 Standard (30 days) + Glacier Deep Archive | ~$1,418/month |
That is roughly a 95% reduction in storage costs.
Why Not CloudWatch Logs?
CloudWatch Logs costs approximately $0.03/GB for ingestion plus $0.03/GB/month for storage. For 10 TB/month:
- Ingestion alone: $300/month
- Storage: $300/month per 10 TB (and growing)
S3 is significantly cheaper at scale, and lifecycle policies handle the archival automatically. CloudWatch Logs also cannot apply S3 lifecycle policies — they are completely separate services.
Why Not AWS Backup?
AWS Backup is designed for broader backup and recovery workflows across multiple AWS services. For simple S3 storage class transitions, lifecycle policies are the native, simpler mechanism. AWS Backup adds unnecessary complexity for this use case.
Key Takeaways
- For high-volume log retention, S3 + Glacier Deep Archive is dramatically cheaper than CloudWatch Logs
- S3 lifecycle policies can handle both transitions (to cheaper storage) and expiration (deletion after retention period)
- One lifecycle rule can define the complete data journey: hot → archive → delete
- You cannot apply S3 lifecycle policies to CloudWatch Logs — they are separate services
- At 10+ TB/month, cost optimization through storage tiering is not optional — it is essential
Never miss a story from us, subscribe to our newsletter