Sources

{ts:357} working with AI powered applications already use go or would like to migrate to go for their AI powered workloads and the most serious challenges developers encountered are related to the the library and documentation ecosystems rather than the core language and runtimes that said the most commonly documented paths for getting started are {ts:373} currently python Centric resulting in many organizations starting AI powered work in Python before moving to a more production ready language the word workload makes me laugh I don't know why I think it's the way you say it but I'm really aggressive when I say it um what you're you're an … {ts:1350} responses in the closed Forum were learning how to write go effectively and verbosity of error handling this matches what we saw in open text form where 11% of responses mentioned learning go learning best practices or issues with documentation as their biggest challenges and another 11% mentioning error handling uh my guess is that

This is a story about the downsides of the Go programming language, the part about it that makes us less productive and our codebases less safe and less maintainable. ... ### Lack of Visibility Modifiers The first root cause of namespace pollution is the lack of visibility modifiers. In what I believe to be an effort to reduce redundant keywords and enhance simplicity, the language designers decided to omit visibility modifier keywords (`public`, `private`, etc…) in favor of symbol naming. Symbols starting with an uppercase letter are automatically public and the rest are private. Sounds like a great choice to promote simplicity. But over time it’s becoming clear that this method has a stronger downside than upside: In most other languages, type names, by convention, begin with an uppercase letter, and variable names begin with a lowercase one. This convention has a very powerful implication — it means that variables can never shadow types. … ``` |1 2 3 4 5 6 7 8 9 10|// type shadowing in Go type user struct { name string } func main() { user := &user{name: "John"} anotherUser := &user{name: "Jane"} // compilation error: user is not a type }| |--|--| ``` This is very common in Go, and I bet most Gophers run into this at some point. In most cases, your coding scopes deal with one user instance, so naming it `user` should be a clear and reasonable choice. However, in Go, whenever you store a private type into a private variable or a public type into a public variable — you run into this. **So you simply start naming your user variables** `u`. … Other problems with lack of native enum support? What about **iterating all possible values** of an enum? That’s something you need every now and then, maybe if you need to send it to the UI for a user to choose one. Not possible. What about **namespacing**? Isn’t it better for all HTTP status codes to reside in a single namespace providing only HTTP status codes? Instead, Go has them mixed with the rest of the public symbols of an HTTP package — like Client, or Server, and error instances. … Initially, I was furious with the designers of the library for not enforcing prevention of such a mistake, “if it happened to me” and everything. Then I thought about it more and realized, they can’t. Go simply does not provide it. At glance, struct literals look like the perfect candidate for config params use case, allowing you to pass in exactly what you need, and omit the rest. But turns out it’s the opposite. … In Go, wrapping each error with another error up the entire call stack, and then garbage collecting all the created objects is almost as expensive. Not to mention the manual coding. If this is what you’re advocating, you’d better off in a try-catch environment, to begin with. Stacked traces can be very useful when investigating errors and bugs, **but the rest of the time they’re highly expensive redundant information**. Maybe a perfect error-handling mechanism would have a switch to turn them on and off when needed. 🤷‍♀️ … This is terrible in terms of performance. But more scary is the fact that it’s not guaranteed. Obviously, `"oops...network error"` can change at any time, and there isn’t any compiler enforcement to help us out. When the author of the package decides to change the message to `"oops...there has been a network error"`, **my error handling logic breaks**, you’ve gotta be kidding me. … In my opinion, the lack of solid community conventions in combination with an ineffective async return value mechanism is the **root cause of terrible coding**, and this is kind of a **standard in the Go community.** [my deepest and sincere apologies to everyone who’s hurt by this paragraph]. Some examples? How about a 400-line function in one of the most popular HTTP frameworks in Go 😨, how about a 100-line function in Google’s gRPC library? … ## Summary If you combine community conventions and naming problems with async return value problems, you end up with **hugely popular libraries** shipping code with complex, 100+ line functions, using one-letter undocumented variables, declared at the other side of the package. This is extremely **unreadable and unmaintainable, and surprisingly common**. In addition, as opposed to other modern languages, Go doesn’t provide any kind of runtime value safety. This results in many **value-related runtime issues** which can easily be avoided.

2025 Go survey of 5,379 developers shows 91% satisfaction but identifies friction with idiomatic patterns, error handling, and module discovery as top challenges. ## Key Points - 91% of Go developers report satisfaction; 65% 'very satisfied' — stable since 2019 - Top three frustrations: idiomatic code patterns (33%), missing language features (28%), trustworthy module discovery (26%) - 13% of respondents new to Go (vs. 21% in 2024), attributed to entry-level hiring decline - 78% not building AI features; 14-point year-over-year decline in AI adoption among survey respondents ... Developers switching between Go and Python/Rust/TypeScript face cognitive load from different idioms—addressing this could improve onboarding and reduce friction for polyglot teams. Source: go.dev

## Terraform in practice We’ve worked with and spoken to platform teams at fast-moving startups, growing mid-size orgs, and multi-cloud enterprises. They all use Terraform and have built internal tooling, standards, and CI pipelines to manage it. And almost all of them still have the same problems: drift, copy-paste modules, fragile reviews, broken dependencies, unclear ownership, and too much tribal knowledge. … ## Problem 1: Terraform is deceptively simple “I just need to provision an S3 bucket and an IAM role. Shouldn’t take more than 10 minutes.” – Famous last words from a junior engineer Terraform starts easy. But what you see is only the top of the HCL iceberg. You write a few resources. Then you need a backend. Then providers. Then variables. Then modules. Then environments. Then workspaces. Then you realize the code is the easy part, and the real complexity is in understanding *what this code will actually do in the real world.* … ## Problem 2: Everyone’s Terraform is different! The beauty of Terraform is its flexibility. The curse of Terraform is also its flexibility. Every company has its own flavors: - Custom modules with their own naming patterns, output conventions, and hidden assumptions - Some teams use `terraform apply`; others have 8-step CI workflows - Some apply per environment, some per resource, some per team - Half the team uses Terragrunt (wrong), the other half doesn’t (also wrong) … ## Problem 3: Drift is inevitable The moment your infrastructure touches the real world, it starts to drift. → An engineer fixes something manually in the AWS console. → Someone disables a policy to unblock a deploy. → An external system updates a tag. → A resource gets destroyed and recreated with new defaults. You run `terraform plan` and it looks fine — because Terraform only knows what *you* told it. It doesn’t know about the out-of-band changes. It doesn’t warn you that a resource has drifted. It happily wipes out your fix and calls it “safe.” … ## Problem 4: The plan looks fine. Until it isn’t. Terraform plans are supposed to bring safety and predictability. In theory, you see what will change before it changes. In practice: You scroll through hundreds of lines, looking for anything suspicious, hoping nothing sneaky gets through. The plan tells you *what* will change. It rarely tells you *why.* It doesn’t tell you what modules depend on this. It doesn’t tell you which services this IAM policy might break. It doesn’t tell you this change will wipe out and recreate a stateful database. … Helpful. You trace through nested modules, try to remember if that variable is a string or a map, and mentally reconstruct the dependency tree. You check the state. You re-run the plan. You eventually just `console.log` your way through `terraform console`. Terraform doesn't make debugging easy. It makes you guess. ## Problem 6: Infra review doesn’t scale In theory, Terraform should enable fast, safe infra changes. In practice, infra PRs sit for days because: - No one has enough context to approve them confidently - Everyone is afraid of breaking something - The person who owns the module is on PTO - The plan is massive, and no one wants to read it The result? You either block product teams or approve blindly. Neither is good. Both are common. ## Some things we haven’t even mentioned yet We haven’t talked about: - Plan noise from computed diffs - Implicit vs. explicit dependencies - Hidden resource recreation - Conditional logic that breaks silently - The tension between DRY and readable code - Running `terraform destroy` in the wrong workspace 😬 ## A better way? If you’ve read this far, you’re probably not surprised by any of this. You’ve lived it. Terraform gives you control, but it doesn’t provide you with confidence. Not when drift is invisible. Not when context lives in someone’s head. Not when reviewing a plan feels like decoding a black box. Infra isn’t hard because we’re doing it wrong. It’s hard because the workflows weren’t designed for how teams actually build and ship today: fast-moving, multi-owner, highly parallel, deeply interconnected.

Even if you're not a Terraform fan, there's a good chance you'll have to support it—especially on projects with a history. Terraform starts to “creak” when the infrastructure becomes really large and multilevel.: **There is no built-in support for multiregionality and multienvironment**. **Managing dependencies between parts of the infrastructure is inconvenient**. **State files without an external backend (S3, GCS, etc.) are a pain**. **It is difficult to share resources between teams without Enterprise solutions**. … - They support a **limited number of cases**, and are often incompatible with existing configs. - They require a separate CLI utility for signing manifests. Opinions in the industry are divided: some find them useful, while others consider them a “crude solution with a good idea.” ... … Positive: - You can use familiar data structures, loops, and conditions. - They are better suited for developers than DevOps engineers. Minuses: - Steep learning curve for ops teams. - Difficulties with typing (especially in CDK). - Not all clouds and resources are supported on par with Terraform. Many teams, having tried the CDK, then return to Terraform due to the inability to easily delegate maintenance to DevOps engineers.

State of Terraform at Scale 2025: Insights from Practitioners ______________________________________________________________________ This report compiles real-world insights from 20+ DevOps practitioners and Terraform users across diverse industries. As Terraform adoption scales, so do the challenges-ranging from environment inconsistencies and state management nightmares to collaboration hurdles and developer experience gaps. The document unpacks these recurring pain points, critiques current tooling, and shares practitioner workarounds-revealing what breaks at scale and what leaders wish they’d known earlier. Key Takeaways … ● Environment Management is brittle: Teams struggle to keep dev/stage/prod consistent without duplicating code or fighting tool limitations (Terragrunt, Workspaces). ● State Management gets messy fast: Monolith vs microstates? Neither is perfect. Cross-repo dependencies and locking issues only get worse at scale. ● Reusable modules are still hard: Teams crave typed, API-like modules-but HCL’s flexibility is a double-edged sword for large teams. ● Collaboration is painful: large teams using Terraform lack proper workflow visibility and often serialize deployments to avoid chaos. ● Validation remains weak: "Best effort" testing leads to accidental infra drifts. Scripts and hacks are common. ● Developer experience matters: Product teams want abstractions and self-service. Many adopt CDKTF or build internal platforms to bridge the gap. Acknowledgment & Thank You … ## Preview text ... developer experience gaps. ... ● Environment Management is brittle: Teams struggle to keep dev/stage/prod consistent without duplicating code or fighting tool limitations (Terragrunt, Workspaces). ● State Management gets messy fast: Monolith vs microstates? Neither is perfect. Cross-repo dependencies and locking issues only get worse at scale. ● Reusable modules are still hard: Teams crave typed, API-like modules-but HCL’s flexibility is a double-edged sword for large teams. ● Collaboration is painful: large teams using Terraform lack proper workflow visibility and often serialize deployments to avoid chaos. ● Validation remains weak: "Best effort" testing leads to accidental infra drifts. Scripts and hacks are common. ● Developer experience matters: Product teams want abstractions and self-service. … ### Introduction #### ______________________________________________________________________ Terraform is a widely adopted Infrastructure as Code (IaC) tool, praised for its declarative approach and extensive provider ecosystem. However, as organizations scale their infrastructure and teams, practitioners often encounter significant challenges that can complicate management and slow down development. Conversations with experienced Terraform users reveal common pain points and highlight areas where existing solutions fall short or where new approaches are desired. … that it "just shifts the goalpost a little bit. It doesn't solve the actual underlying problem". ● Another approach involved using Terraform Workspaces for different environments, but this was found to be "really weirdly" handled by Terraform, at least in one user's experience. ● Many companies ultimately resort to having totally different Terraform templates for … is perceived as a "slowly changing beast," and avoid it for faster-moving components. "I will never put, you know, any fast moving stuff in Terraform because it's difficult to manage," confessed one user. ● Users desire a solution that provides a summary view of all repositories, pipelines, and statuses, with the ability to selectively run them. … ### Validation and Testing Deficiencies #### ______________________________________________________________________ Ensuring that deployed infrastructure matches the intended configuration and functions correctly is challenging, and practitioners feel Terraform's built-in validation and testing tools are not fully mature. … "Whether it is functioning as intended is a totally different question," one practitioner stressed, despite acknowledging that Terraform's own testing features help validate the plan's intent. ● Some teams have had to resort to embedding validation scripts or commands within their Terraform code, often using local provisioners or depends_on to ensure they run … because "when people generate a plan, it would start saying that no one has to apply because there is a tri block there," making the plan harder to validate. This happens because try is evaluated at the apply phase if values are not statically assigned. ● More built-in support for robust testing and validation within Terraform is desired. Terraform is trying to address this with tests, but it's "still not really mature".

### 1. State Management: The Double-Edged Sword The Terraform state file (`terraform.tfstate`) is the heart of Terraform. It's a JSON file that maps your code to real-world resources. This is how Terraform knows what it's managing. But it's also its biggest source of pain. - **The Problem:** The state file is a single source of truth that can become a single point of failure. If it gets corrupted, lost, or out of sync, Terraform loses its "memory," leading to chaos. - **The Impact:** Manually editing the state file is terrifying and error-prone. Concurrency issues arise when multiple people run `terraform apply` at the same time, leading to state corruption. And by default, state is stored locally, which is a non-starter for teams. … ### 2. Refactoring is Painful and Risky As your infrastructure evolves, your code needs to evolve with it. You'll want to rename resources for clarity, move them into modules, or reorganize your file structure. In a normal programming language, this is a simple refactor. In Terraform, it's a destructive operation. - **The Problem:** If you rename a resource in your `.tf` file (e.g., from `aws_instance.web` to `aws_instance.web_server`), Terraform sees one resource to be destroyed and one new resource to be created. - **The Impact:** This can cause catastrophic downtime and data loss for stateful resources like databases or storage buckets. … - **The Problem:** While major cloud providers are excellent, smaller or community-led providers can be buggy, lack features, or lag behind API updates. You're entirely dependent on the provider's implementation. - **The Impact:** You might find a bug where `terraform plan` shows no changes, but `apply` fails. Or a new cloud service is released, and you have to wait months for the provider to support it. … - **The Problem:** Terraform has no native, end-to-end secret management solution. The state file itself can contain sensitive values in plain text after an `apply`. - **The Impact:** Accidentally committing a `.tfvars` file with secrets or having an exposed state file can lead to a severe security breach. … - **The Problem:** Terraform needs to refresh the state of every resource in your configuration by making API calls to your cloud provider. For large setups, this can take many minutes. - **The Impact:** Long feedback loops kill developer productivity and make quick fixes anything but quick. … - **The Problem:** There's no built-in testing framework. Unit testing HCL is difficult, and integration testing (spinning up real infrastructure) is slow, expensive, and complex to manage. - **The Impact:** It's easy for bugs to slip into production, causing outages or security vulnerabilities. Confidence in making changes decreases as the infrastructure grows. … ### 9. Cryptic Error Messages While this has improved significantly in recent versions, Terraform can still produce error messages that are baffling, especially when dealing with complex modules or provider bugs. … - **The Problem:** Terraform only detects drift when you run a `plan` or `apply`. It doesn't have a built-in, continuous monitoring system to alert you when drift occurs. - **The Impact:** Your state file no longer represents reality, and the next `apply` could have unintended, destructive consequences by trying to "fix" the manual change.

Terraform's error messages are usually cryptic and hard to understand. Finally, when Terraform fails, it just drops the deployment and leaves the half-deployed resources as is. There is no built-in way to revert to a last good state. Supports a wide range of providers, such as cloud platforms and all sorts of server-based software such as Hashicorp Vault, Grafana, etc. ... Cryptic error messages The quality of the documentation varies a lot, but generally speaking it doesn't go into enough details Terraform sometimes fails for obscure reasons Terraform sometimes updates resources that have no changes No built-in ability to roll back to a previously working state if the deployment fails Terraform is no longer FOSS Weird configuration language Renaming resources usually leads to painful problems Some limitations prevent the efficient use of Terraform in a multi-environment setup (which is usually the case), which lead to the birth of Terragrunt to overcome those limitations. … 1. There was generally one example on each resources in terraform documentation which makes understanding a bit challenging. 2. There are very few developers with the terraform experience. 3. After writing the terraform scripts, the developers has to check the terraform plan properly before proceed the terraform apply command . There is a possibility that Developers run terraform apply command directly which would lead to deletion or modification of the resources and once resource is modified or deleted, there is no way to get the resources back. … Basically whenever your DevOps engineers are overseeing in excess of ten machines or when you need numerous groups not zeroed in on DevOps to assist with claiming the framework facilitating their code. Prominent sentiment is that Terraform isn't exceptionally secure, fight tried, and spilling mysteries happen effectively on mishap. Thus, Terraform is not so great when you need to store bunches of touchy mysteries that your organization is lawfully needed to watch in case it is the finish of you. … The actual language is somewhat surprising and this makes it difficult for new clients to get onboarded into the codebase. While it's improving with later deliveries, essential ideas like "map a variety of choices into a bunch of designs" or "apply this rationale in the event that a variable is indicated" are conceivable however superfluously unwieldy. … One of my main problems with Terraform is when I'm trying to delete things and I end up with dependencies blocking the deletion. Sometimes I need to manipulate state and am left with orphaned resources. When things don't work, it's tricky to troubleshoot. Really positive, we have the majority of our infrastructure represented as code which makes deployments and maintaining our infrastructure easier. … Looping strategies like for_each are rather complex to understand when you are new to terraform. It does not have a rollback feature & if something fails all the changes before that failed change would still be applied. Some conditional logics are unnecessarily cumbersome. Also overally analytics for users running plan & apply is missing, it's better for management purpose & debugging the plan which might have caused an infra issue.

Even experienced teams can run into predictable issues when adopting Terraform. Below are common pitfalls, why they happen, and practical fixes to avoid downtime and confusion. Poor state management • Why it happens: Teams keep state files locally or don’t enable locking, causing conflicts and state corruption. • Solution: Use remote state backends with locking (often supported by cloud object stores or managed services). Be careful when migrating state and avoid manual edits unless necessary.

Terraform hits hard when your infrastructure grows faster than your control over it. You run `terraform apply`, and the plan looks fine—until the change breaks something you didn’t expect. This is the core frustration: Terraform’s strength in managing large, complex clouds also exposes sharp edges when your state, modules, and workflows drift out of sync. The first pain point is state management. Remote state is supposed to solve collaboration issues, but locking, version conflicts, and backups can slow teams down or block deployments outright. Every mismatch between real resources and recorded state becomes a delay, a risk, and a source of hidden cost. The second is module complexity. Terraform encourages reusable modules, but deep dependency chains and over-abstracted components make debugging painful. Changing one variable in a shared module can trigger unrelated updates in production. Simple fixes can trigger large-scale plans, making rollbacks harder. Third, drift detection remains weak. Terraform can detect most changes, but out-of-band updates often slip through until something breaks. Manual audits and refresh commands add friction. Accurate drift detection should be automated and safe, but in practice, it is another manual chore. Lastly, execution speed matters. Large plans on multi-cloud setups run slow. Waiting minutes or hours to see if your change worked kills momentum. Quick feedback loops are rare, and workarounds often involve hacks to split plans or bypass certain checks.

div Terraform Errors are more common than most teams realize. While terraform has become the IaC tool of choice for many organizations. The reality is that Terraform makes it deceptively easy to get started but considerably more challenging to get right. Many teams discover this only after they’ve accumulated significant technical debt. Simple deployments can quickly become maintenance nightmares when you overlook best practices. … ### Adopt Trunk-Based Development for Better Terraform Collaboration ... However, unlike application code, infrastructure can have only one version deployed. Keeping multiple long-lived branches in a Terraform repository is not common practice. ... … ## 2. Terraform Error: Ignoring Modules in Your Infrastructure Without modules, lengthy and duplicated code appears across multiple environments as developers copy and paste configurations rather than reusing established patterns. It can cause inconsistencies across environments, and making a simple change would require updates in multiple places. Modules help keep provider versioning such as Terraform AWS provider or Terraform Azure provider consistent across your configuration. … ## 3. Not Pinning Provider Versions: A Common Terraform Pitfall When you don’t specify exact provider versions, Terraform automatically pulls the latest version during initialization, which can lead to unexpected behaviour or broken deployments when providers release breaking changes. Here is the right way: … ## 4. Terraform Mistake: Poor Resource DependenciesorTerraform builds its dependency graph based on explicit references between resources. But some dependencies exist at runtime that aren’t visible in configuration. Failing to declare these “hidden” dependencies can lead to subtle, hard-to-debug issues where resources are technically created but don’t function properly together. The example below shows why Terraform can miss important runtime dependencies and how `depends_on`  can be used to fix it: … ## 7. Terraform Errors Caused by Inconsistent File Structurese.One of the most common Terraform Errors teams make is cramming numerous resources, data sources, and variables into a single monolithic .tf file. This approach might seem convenient initially, but as your infrastructure expands, it becomes increasingly difficult to navigate, troubleshoot, and collaborate effectively. A well-structured Terraform project typically includes several specialized files, each with a distinct purpose.

# 10 Biggest Pitfalls of Terraform Terraform, a popular tool in Infrastructure as Code (IaC), faces challenges with managing multiple environments and scaling, leading to complex and error-prone setups. Terramate, a new CLI tool, addresses these issues by introducing stack concepts, global variables, and code generation to simplify environment management and reduce code complexity. It improves upon Terraform's limitations in versioning, backend configuration, and resource lifecycle management, offering a more streamlined and flexible approach for complex infrastructure management. Terramate's innovative features enhance efficiency and control, making it a valuable addition to the IaC toolset. Terraform (or OpenTofu if you prefer open source) has emerged as a pivotal player in the evolving Infrastructure as Code (IaC) landscape, facilitating the management and provision of cloud resources through code. However, like any tool, it has drawbacks and tradeoffs. Challenges such as **managing multiple environments with workspaces**, **maintaining module versions** and **backend configurations**, and ** managing resource lifecycles** often make Terraform code hard to read and prone to errors. Moreover, scaling can be cumbersome due to a lack of stack concept, leading to complications in more intricate environments. … ## 1. Terraform Workspaces Terraform Workspaces help you manage different environments, like staging, development, and production. However, they can be tricky to handle. For example, the code can be difficult to understand because you have to use the `count` parameter a lot to create resources based on conditions. Also, it gets harder when you want to scale or grow with Terraform Workspaces because you need to add more connections between them when managing different environments. … ## 2. Maintaining Module Versions In Terraform, a feature called the module block lets users use pre-set modules. But there's a problem with this block. The `source` and `version` attributes in this block, which are used to specify where the module comes from and which version of the module to use, don't allow for variable interpolation. Variable interpolation is replacing a placeholder in a string with its actual value. This limitation can cause trouble when you're trying to set up modules in a flexible or dynamic way. … ## 3. Hardcoding Backend Configuration When you’re working with Terraform, you might need to make copies of Root Modules, but this can cause unexpected problems if you’re not careful with the backend configuration. The backend configuration is where Terraform stores information about your infrastructure. If you copy the backend configuration without changing the `key` or … ## 7. Missing Stack Concept Terraform is unique in the world of IaC tools because it doesn’t have a stack concept. A stack is a collection of resources that are managed together. Instead, Terraform only focuses on what’s happening within a single directory, a root module. This can cause problems when dealing with bigger, more complex environments because it’s not designed to handle multiple collections of resources at once. … ## 8. Code Duplication In Terraform, when you want to use a module (which is a pre-set piece of code) multiple times, you have to copy the call to the module and the arguments (the specific instructions you give to the module) each time. This leads to repeated code, making your codebase larger and harder to maintain. Terramate offers a solution to this problem with its Code Generation feature. This feature creates the necessary blocks of code based on your configurations. This means you don’t have to repeat the same blocks of code multiple times, which reduces code duplication and makes your code more efficient. # 9. Monostacks If you’re managing a lot of resources (like virtual machines, databases, etc.) in Terraform, it can cause some problems. For example, if something goes wrong, it could affect many of your resources (this is known as a “big blast radius”). Also, executing plans and applying changes can take a long time when dealing with many resources. Additionally, if there are discrepancies or “drifts” in a single resource, it can prevent you from applying new changes. … ## 10. Deep Merging of Maps and Objects In Terraform, merging or combining maps and objects at multiple levels, also known as “deep merging”, is not allowed. A map is a collection of key-value pairs, and an object is a complex structure containing multiple data types. This limitation makes it hard to merge default configurations with user inputs. For instance, it’s difficult to create keys or attributes that conflict, and changing the value of an attribute in a nested structure is impossible. … ## Conclusion Terraform has played a key role in popularizing the concept of Infrastructure as Code, where you manage your IT infrastructure using code. However, it’s not without its challenges. These include issues like code that is hard to read, difficulty scaling with workspaces, problems maintaining versions of modules, the need to hardcode backend configurations and the complexity of managing the lifecycle of resources.