www.allthingsdistributed.com
In S3 simplicity is table stakes - All Things Distributed
Excerpt
## Performance matters Over the years, as S3 has evolved from a system primarily used for archival data over relatively slow internet links into something far more capable, customers naturally wanted to do more and more with their data. This created a fascinating flywheel where improvements in performance drove demand for even more performance, and any limitations became yet another source of friction that distracted developers from their core work. … If anything, I expect this trend to accelerate as customers pull the experience of using S3 closer to their applications and ask us to support increasingly interactive workloads. It’s another example of how removing limitations – in this case, performance constraints – lets developers focus on building rather than working around sharp edges. ## The tension between simplicity and velocity The pursuit of simplicity has taken us in all sorts of interesting directions over the past two decades. There are all the examples that I mentioned above, from scaling bucket limits to enhancing performance, as well as countless other improvements especially around features like cross-region replication, object lock, and versioning that all provide very deliberate guardrails for data protection and durability. ... … But on the other hand, racing to release something with painful gaps can frustrate early customers and worse, it can put you in a spot where you have backloaded work that is more expensive to simplify it later. This tension between simplicity and velocity has been the source of some of the most heated product discussions that I’ve seen in S3, and it’s a thing that I feel the team actually does a pretty deliberate job of. … The other complexity is that because these tables are actually made up of many, frequently thousands, of objects, and are accessed with very application-specific patterns, that many existing S3 features, like Intelligent-Tiering and cross-region replication, don’t work exactly as expected on them.
Related Pain Points
S3 performance limitations strain developer productivity
7As S3 usage evolved from archival to interactive workloads, performance constraints became friction points that distract developers from core work. Limitations force developers to implement workarounds rather than focus on building features.
S3 features don't work as expected with table-based workloads
6Existing S3 features like Intelligent-Tiering and cross-region replication have unexpected behavior when tables are stored as thousands of objects with application-specific access patterns, limiting their effectiveness.
Tension between S3 simplicity goals and feature completeness
5Releasing S3 features with painful gaps frustrates early customers and creates expensive technical debt for simplification later. The pursuit of both simplicity and velocity creates friction in product development.