How I Saved 95% of AWS S3 Cost
Last year, I noticed a steady increase in my AWS charges. After paying for a few monthly bills over $50, I decided to track down what contributed most to the cost, and be Frugal.
That was pretty straightforward – I just need to check cost explorer. And quickly I found that S3 contributed to 76% of total cost. So far, all I did was just a few clicks.
The next steps would be identifying cost-saving opportunities. That did require some Deep Dive, but not too bad – I found 2 aspects where I’m using more than what I really need.
Versioning
S3 Versioning provides the capability to keep multiple variants (or “versions”) of objects. For my use case, I have a few buckets with versioning enabled that stores automated daily backups. The backup tool I used is configured to keep a fixed number of recent backups and will delete the older ones. However, as the buckets has versioning enabled, the older objects are not effectively “deleted”, but was added a delete marker instead (reference), and keep incurring storage costs. As the backup tool is creating new daily backups (~ 3 GB) at the same time, it cause the cost to going up steadily.
I then decided to suspend versioning, and delete all older objects, as I don’t need them.
Storage Classes and Lifecycle
Another opportunity I noticed was that, most of my objects are accessed infrequently. For me, my main use cases are:
- programmatic backups (for blog, NAS)
- media files for blogs
- emails to my custom domain to be forwarded
- data of my side projects
- personal files
- logs, and other files for AWS internal housekeeping
Among those, 1 is infrequently, if ever, accessed. 2 and 4 are frequently accessed, or, to be more precise, require low SLA for accessing. For rest of use cases, chances are that objects will be less frequently accessed over time.
By default, objects uploaded to S3 are in Standard class, which has the best performance but highest storage cost as well. Apparently, this is overkill for my use case. So I looked at different storage classes and configured use case-specific lifecycle rules.
Intelligent-Tiering
Intelligent-Tiering is “smart” solution that optimize costs by automatically moving data to the appropriate storage class based on access pattern, and has no impact on performance. This is almost a no-brainer if you are looking for a “one-size fits all” solution.
For me, I decided to keep my blog media files and side project related data in this storage class, since they may be programmatically accessed (even though not frequently) at any random time. Besides, I have also set up lifecycle rules to move most objects to this storage class, before eventually moving to an even less frequently accessed storage class, such as Glacier storage classes.
Glacier storage classes
Glacier storage classes, including Glacier Instant Retrieval, Glacier Flexible Retrieval and Glacier Deep Archive, are designed for archival use cases. While they have lower costs for storage, they are associated with a retrieval charge, and higher SLA to retrieve the objects.
In my case, the one that I used most is Glacier Instant Retrieval. I move my NAS backups to this storage class, and for personal files, they are also transitioned to this storage class after staying in Intelligent-Tiering for a period of time.
There are other storage classes such as S3 Standard – Infrequent Access and S3 One Zone – Infrequent Access, which I didn’t use a lot. More information can be found in AWS S3 official docs:
How much I saved
Daily cost by usage type
The storage cost for standard storage class dropped drastically, due to the deletion of old version of backup objects, as well as objects that are moved to other storage classes – which accounted for the increase of “USW2-Requests-Tier3” charge.
The daily coast decreased from $1.38 to $0.05 (about 95%).
Monthly cost by usage type
From monthly aggregation view, the standard storage class contributed to a smaller portion of total cost.
Current status
Currently Standard storage class contributed to most number of objects, while Glacier storage classes contributed to more storage. With that, my current monthly S3 cost is around $3 which is about $0.10 per day.
Final Thoughts
AWS has some analytical tools available to help identifying cost saving opportunities. Check out AWS Cost Explorer, or S3 Storage Lens.