How 170 MB cost me $350 in 813 million S3 requests
I had wanted to see the analytics for this blog for quite some time. Turning on CloudFront access logs from the very start was a conscious decision: I wanted analytics, but I did not want to reach for Google Analytics - too much CSP complexity, the privacy concerns, the cookie policy, all of it. Logging to my own S3 bucket sidesteps every one of those, and I knew I would one day build something around the data. In practice that "something" was me running an ad-hoc Athena query from time to time to see how many hits /blog/* got in a given month - which works, but is not exactly user friendly. I had already built two dashboards (one for DMARC reports and one for email delivery), so I figured a third would be no trouble.
Adding it meant backfilling about a year of CloudFront access logs. The whole dataset is around 170 MB. Backfilling it cost me roughly $350. Done right it would have cost about $3. The data was tiny, the mistake was tiny, and the bill was not - and the reason is a single architectural decision I did not think twice about.