The Problem
You host a large public dataset in S3: genomic reads, satellite imagery, machine learning training data. Thousands of researchers pull terabytes every month. Your storage bill is fine. Your data transfer bill is not.
You want to share the data openly, but you refuse to subsidize every download for every stranger on the internet.
The Solution
Turn on S3 Requester Pays on the bucket. The requester pays for the request costs and the data transfer. You, the bucket owner, still pay only for storage.
Requesters must be authenticated AWS users and must send an x-amz-request-payer: requester header (or --request-payer requester on the CLI) to acknowledge the charge.
How It Works
The Billing Split
| Cost | Who Pays |
|---|---|
| Storage (GB-month) | Bucket owner |
| Request charges (GET, LIST, etc.) | Requester |
| Data transfer OUT to the internet | Requester |
Anonymous requests are rejected. Requester Pays only works with authenticated AWS requesters, because AWS needs an account to bill.
Enabling Requester Pays
1
2
3
aws s3api put-bucket-request-payment \
--bucket public-genomics-dataset \
--request-payment-configuration '{"Payer":"Requester"}'
Verify:
1
aws s3api get-bucket-request-payment --bucket public-genomics-dataset
That flips the switch. From now on, any request without the acknowledgement header returns 403 Access Denied.
Downloading From a Requester Pays Bucket
CLI:
1
2
3
4
5
aws s3api get-object \
--bucket public-genomics-dataset \
--key samples/chromosome-17.fastq.gz \
--request-payer requester \
chromosome-17.fastq.gz
boto3:
1
2
3
4
5
6
7
8
9
import boto3
s3 = boto3.client("s3")
s3.download_file(
Bucket="public-genomics-dataset",
Key="samples/chromosome-17.fastq.gz",
Filename="chromosome-17.fastq.gz",
ExtraArgs={"RequestPayer": "requester"}
)
Miss the header, and S3 rejects the call. That is what prevents accidental charges.
Bucket Policy Considerations
Requester Pays does not control who can read the bucket. You still need a bucket policy or ACL that grants read access. Requester Pays only controls who pays; you decide separately who is allowed to download.
Combine Requester Pays with a bucket policy scoped to authenticated principals:
1
2
3
4
5
6
7
8
9
10
11
12
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": { "AWS": "*" },
"Action": ["s3:GetObject", "s3:ListBucket"],
"Resource": [
"arn:aws:s3:::public-genomics-dataset",
"arn:aws:s3:::public-genomics-dataset/*"
]
}]
}
Any authenticated AWS caller can now list and read, and they cover the cost of doing so.
What Requester Pays Does Not Support
- Anonymous access: every request must be signed with SigV4
- BitTorrent: deprecated anyway
- Static website hosting: the website endpoint does not send the request-payer header
If you need anonymous public download, Requester Pays is the wrong tool.
Why Not the Alternatives?
Keep the bucket public and eat the transfer cost: Fine at gigabyte scale, expensive at terabyte scale. A viral dataset can generate a five-figure monthly bill.
Move the data to CloudFront: Reduces per-GB cost slightly and gives edge caching, but you still pay for the traffic. Good for latency, wrong for cost-shifting.
Force users to submit a signed data-sharing agreement: Adds friction, does not solve the bill. Requester Pays achieves the same billing model with a header flip.
Distribute on the AWS Open Data Sponsorship Program: Works, but requires AWS to accept your dataset into the program. Requester Pays needs no approval.
Key Takeaways
- Requester Pays shifts request and transfer costs to the downloader; the owner still pays for storage
- Requesters must send
x-amz-request-payer: requester; anonymous access is not supported - You still need a bucket policy or ACL granting read access. Requester Pays only sets the payer
- This is the standard pattern for sharing large scientific or archival datasets on S3
- Do not combine with static website hosting or BitTorrent. The model does not apply
Never miss a story from us, subscribe to our newsletter