Stripe's monorepo developer environment - Made of Bugs

### Enabling new dependencies 🔗︎ Stripe was continually adding and reworking dependencies for the API or other services; for instance, implementing rate limiting in the API required a Redis cluster. In a world where development code runs on laptops, this meant that every laptop would now need a Redis installed and potentially configured. Updating configuration on developer laptops was challenging, and often this would be left to word-of-mouth between engineers: “Hey, I pulled … invocation to install Redis might get slipped into a random script that developers run regularly, in the hopes of resolving the issue, which was also brittle and periodically caused random tools to be slowed down. With the devbox model, the team adding Redis was already responsible for configuring it in production, and they could add appropriate Puppet configuration to ensure it was installed and running on devboxes, as well. In the event that a user ran into problems with this new path, members of that team, or of a devprod, would be able to … ## Editors and source control 🔗︎ Before you can run new code under development, you need to actually access code via source control and edit it. At Stripe, even though code **ran** in the cloud, git checkouts and editors lived locally, on developer laptops. Stripe settled on this approach for a number of reasons, including: - Support for a wide variety of editor and IDE environments … ) support running a local UI against a remote filesystem, but many do not. By keeping code on the laptop, Stripe allowed developers to continue using any editor they wanted. - Latency Stripe had developers around the globe, but at the time did not yet maintain substantial infrastructure outside of the US. Editing code on the other side of a 200+ms network latency is **painful**. Standing up devboxes globally was an option but would have been complicated for numerous reasons. - Source code durability and filesystem performance By keeping the source-of-truth on laptops and off of the devbox, it was much easier to treat the execution environment as ephemeral and transient. That property, in turn, was very helpful for keeping environments up-to-date and preventing drift over time. … ### Automatic synchronization 🔗︎ Keeping code on laptops, and executing in the cloud, though, poses a new challenge: How does code, once edited, **get** from the laptop onto the devbox? Even before I joined, Stripe had a “sync” script that glued together a file-watcher with … On the whole, these investments largely succeeded: most developers were able to treat code sync as simply a “fact of life,” and rarely had to think about or debug it. One downside of synchronization is that it makes it harder to write “code that acts on code,” such as automated migration tools, or even just linters or code-generation tools (Stripe ended up relying on a handful of code-generated components which needed to be checked in for various reasons). This remained a bit of a pain point through my tenure; we made do with a mix of strategies: - Run on the developer laptop, and deal with the environment challenges - Run on the devbox and then somehow “sync back” generated files. We had a small protocol to run scripts in a “sync-back” wrapper, s.t. they could ask for file to be copied back to the laptop, but it remained somewhat awkward and non-ergonomic and occasionally unreliable. … Moreover, for various historical, business, and technical reasons, Stripe’s codebase was fairly tightly coupled in a number of ways; it was clear to those of us on the developer productivity team that it would not easily or rapidly be decomposed in any way (e.g. into a larger number of microservices). We did continually invest in tools and patterns for modularity and abstraction within the monorepo, but the confidence in the overall “stickiness” of the structure of the Ruby monorepo also justified our large investments in specialized tooling.