Back

news.ycombinator.com

I think in general Python's biggest challenge is that it doesn't scale ...

11/12/2020Updated 8/25/2025
https://news.ycombinator.com/item?id=25073308

I strongly suspect that devs' satisfaction with Python is strongly correlated with the size of the codebase they're working on. Generally people using Python for one-off projects or self-contained tools tend to be pretty happy. People stuck in sprawling enterprise codebases, with O(million) lines of code to wrangle, seem almost universally miserable with the language. What I've observed a lot is that many startups or greenfield projects start with Python to get an MVP out the door as fast as possible. Then as the scope of the software expands they feel increasingly bogged down and trapped in the language. reissbaker on Nov 13, 2020 ... I have used that library on multiple projects for my job. It makes the code run about 50% slower, on average, because all the type checking is done at run-time. I am OK with the slow down because I don't use Python when I need speed. My "developer speed" was greatly improved with stricter types. … If we are being realistic, most of the time 80% of python programs are spend waiting on network. Granted, python is not overly fast, but then most of the time your bottleneck is the developer not the language. > no concurrency Yes, this is a pain. I would really like some non GIL based threading. However its not really been that much of a problem. multiprocessing Queues are useful here, if limited. Failing that, make more processes and use an rpc system. > typing as second-class citizens The annotation is under developed. being reliant on dataclass libraries to enforce typing is a bit poop. > People stuck in sprawling enterprise codebases, with O(million) lines of code to wrangle, seem almost universally miserable with the language. I work with a _massive_ monorepo. Python isn't the problem, its programmer being "clever" or making needless abstractions of abstractions. None of that is python's issues, its egotistical programmer not wanting to read other people's (un documented) code. And not wanting to spend time make other people's code better. … As for processing scalability - Python is OK, but it's considerably hampered by BDFL's own opinions. The result is a few third party libraries that implement parallelism in their own way. That functionality should be integral to the standard library already. The worst part is lack of standard API for data sharing between processes. > packaging Python's packaging issues only start with package management. Setuptools is a wholly mess of a system, that literally gave me headaches for the lack of "expected features". I hate it with every single cell in my body. And then there are systems and libraries, where you literally cannot use docker (Hello PySpark!). … If only for the lack of (strong/static) typing and the relatively underpowered control flow mechanisms (e.g. Python often using exceptions in their stead)... While surely all languages have pain points that show up at scale, Python still has a notable lot of significant ones precisely in this area. >docker, poetry, venv... … Ever wonder why you don't hear endlessly about different ways of doing dependency management in non-Python ecosystems? Because we have tools that actually work, and get on with actually writing programs. It baffles me that Python keeps making new tools and keeps repeating the same mistakes over and over: non-reproducible dependency resolution, excessively tight integration between the language and the build tools, and tools and infrastructure that can't be reused locally. … 1) system packages: almost always out of date for my needs 2) Binary wheels: I actually haven't investigated this much, maybe it will work (and if it does, I'll buy you a drink if we ever meet in person). 3) Building from source: this kinda proves my point about Python having poor dependency management tools if this is a serious response. In general, this would be much further down the rabbit hole than I want to go. … ... Additionally, they are part of a larger application, which is mostly managed by pip, which means that I need both pip and conda which is where things get really, really hairy. I actually blame Google and FB here, as neither of them use standard python dependency management tools, and many of their frameworks bring in the world, thus increasing the risk of breakage. … What I've observed a lot is that many startups or greenfield projects start with Python to get an MVP out the door as fast as possible. Then as the scope of the software expands they feel increasingly bogged down and trapped in the language.

Related Pain Points6

Ecosystem fragmentation and dependency management chaos

8

PyPI security breaches forced strict corporate policies, fragmented package management (pip/conda), and critical libraries like NumPy and Pandas struggle with GPU demands, creating incompatible forks and version conflicts.

dependencyPythonPyPIpip+3

Lack of standard API for inter-process data sharing

6

Python lacks a standard, well-designed API for sharing data between processes. The multiprocessing.Queues are limited and third-party libraries implement parallelism in inconsistent ways, forcing developers to build custom RPC solutions or choose between incompatible approaches.

architecturePythonmultiprocessing

Most developers stuck on older Python versions despite major performance gains

5

83% of developers run Python versions 1+ years old despite containerization making upgrades trivial. Missing out on 11-42% performance improvements (3.11→3.13/3.10→3.13) and 10-30% memory reductions without code changes.

dxPythonDocker

Project complexity balloons quickly with scripting-style codebases

5

Python scripts written without discipline grow unwieldy and difficult to maintain. Historic cross-implementation compatibility breaks regularly, causing pain. Refactoring becomes risky without strong static analysis.

architecturePython

Docker incompatibility with certain Python frameworks

5

Some Python libraries and frameworks (notably PySpark) cannot be easily used with Docker, forcing developers to choose between containerization approaches or framework selection, limiting deployment flexibility.

deployPythonDockerPySpark

Runtime type checking overhead without compile-time guarantees

4

Libraries that provide runtime type checking in Python add approximately 50% performance overhead while only catching errors at runtime. This creates a false trade-off where developers get stricter type checking but lose performance without gaining the compile-time safety of statically-typed languages.

performancePython