www.allthingsdistributed.com

In S3 simplicity is table stakes - All Things Distributed

3/14/2025Updated 3/17/2026

Excerpt

## Performance matters Over the years, as S3 has evolved from a system primarily used for archival data over relatively slow internet links into something far more capable, customers naturally wanted to do more and more with their data. This created a fascinating flywheel where improvements in performance drove demand for even more performance, and any limitations became yet another source of friction that distracted developers from their core work. … If anything, I expect this trend to accelerate as customers pull the experience of using S3 closer to their applications and ask us to support increasingly interactive workloads. It’s another example of how removing limitations – in this case, performance constraints – lets developers focus on building rather than working around sharp edges. ## The tension between simplicity and velocity The pursuit of simplicity has taken us in all sorts of interesting directions over the past two decades. There are all the examples that I mentioned above, from scaling bucket limits to enhancing performance, as well as countless other improvements especially around features like cross-region replication, object lock, and versioning that all provide very deliberate guardrails for data protection and durability. ... … But on the other hand, racing to release something with painful gaps can frustrate early customers and worse, it can put you in a spot where you have backloaded work that is more expensive to simplify it later. This tension between simplicity and velocity has been the source of some of the most heated product discussions that I’ve seen in S3, and it’s a thing that I feel the team actually does a pretty deliberate job of. … The other complexity is that because these tables are actually made up of many, frequently thousands, of objects, and are accessed with very application-specific patterns, that many existing S3 features, like Intelligent-Tiering and cross-region replication, don’t work exactly as expected on them.

Source URL

https://www.allthingsdistributed.com/2025/03/in-s3-simplicity-is-table-stakes.html

Related Pain Points