The Problem
You store call transcript files monthly. Users access them randomly within the first year (unpredictable patterns), then infrequently after one year. You need users to:
- Query and retrieve files less than 1 year old quickly
- Accept a delay for retrieving older files
The Solution
Use S3 Intelligent-Tiering for the first year (handles unpredictable access with automatic optimization), then a lifecycle policy to Glacier Flexible Retrieval after one year. Query recent data with Amazon Athena and archived data with S3 Glacier Select.
How It Works
Architecture
1
2
Year 1: S3 Intelligent-Tiering → Query with Amazon Athena (instant)
Year 2+: S3 Glacier Flexible Retrieval → Query with Glacier Select (minutes to hours)
Configuration
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# Set up Intelligent-Tiering
aws s3api put-bucket-intelligent-tiering-configuration \
--bucket call-transcripts \
--id "TranscriptTiering" \
--intelligent-tiering-configuration '{
"Id": "TranscriptTiering",
"Status": "Enabled",
"Tierings": []
}'
# Lifecycle to Glacier after 1 year
aws s3api put-bucket-lifecycle-configuration \
--bucket call-transcripts \
--lifecycle-configuration '{
"Rules": [
{
"ID": "ArchiveAfter1Year",
"Status": "Enabled",
"Filter": {},
"Transitions": [
{
"Days": 365,
"StorageClass": "GLACIER"
}
]
}
]
}'
Querying Each Tier
Recent data (Intelligent-Tiering) — Use Amazon Athena for serverless SQL queries directly against S3:
1
2
3
SELECT * FROM call_transcripts
WHERE call_date > date_add('year', -1, current_date)
AND customer_id = '12345';
Archived data (Glacier) — Use S3 Glacier Select to run queries on archived objects without restoring the entire file:
1
2
3
4
5
6
7
aws glacier initiate-job \
--vault-name call-archive \
--job-parameters '{
"Type": "select",
"Expression": "SELECT * FROM archive WHERE customer_id = '\''12345'\''",
"OutputLocation": {"S3": {"BucketName": "results", "Prefix": "query/"}}
}'
Why Intelligent-Tiering for Year 1?
The access pattern within the first year is random and unpredictable. Some transcripts get accessed frequently (ongoing investigations), others are never touched. Intelligent-Tiering automatically moves untouched objects to the infrequent access tier after 30 days, saving costs with no retrieval fees when patterns shift.
Why Not the Alternatives?
S3 Glacier Instant Retrieval for everything — High retrieval fees make it expensive for random, unpredictable access during the first year.
S3 Standard with RDS for metadata — Requires managing an RDS database. Athena is serverless and queries S3 directly, eliminating infrastructure management.
S3 Standard with Glacier Deep Archive — Deep Archive has retrieval times of 12-48 hours. Glacier Flexible Retrieval (3-5 hours standard, 1-5 min expedited) is a better fit when “delay is acceptable” rather than “delay of hours.”
Key Takeaways
- Intelligent-Tiering is ideal for the “active but unpredictable” phase — no retrieval fees, automatic optimization
- Lifecycle to Glacier Flexible Retrieval is right for the “rarely accessed, delay acceptable” phase
- Amazon Athena queries S3 data with serverless SQL — no infrastructure to manage
- Glacier Select allows querying archived data without full restoration
- This two-stage pattern (Intelligent-Tiering → Glacier) is common for data with distinct active and archive phases
Never miss a story from us, subscribe to our newsletter