HN Reader

The author gets close to what I think the root problem is, but doesn't call it out.

The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc).

Meanwhile, go showed us what good green threads can look like. Then java did it too. Meanwhile, js had better async support the whole time. But all it did was show us that async code just plain sucks compared to green thread code that can just block, instead of having to do the async dances.

So, why engage with it when you already had good solutions?

I learned about the concept of async/await from JS and back then was really amazed by the elegance of it.

By now, the downsides are well-known, but I think Python's implementation did a few things that made it particularly unpleasant to use.

There is the usual "colored functions" problem. Python has that too, but on steroids: There are sync and async functions, but then some of the sync functions can only be called from an async function, because they expect an event loop to be present, while others must not be called from an async function because they block the thread or take a lot of CPU to run or just refuse to run if an event loop is detected. That makes at least four colors.

The API has the same complexity: In JS, there are 3 primitives that you interact with in code: Sync functions, async functions and promises. (Understanding the event loop is needed to reason about the program, but it's never visible in the code).

Whereas Python has: Generators, Coroutines, Awaitables, Futures, Tasks, Event Loops, AsyncIterators and probably a few more.

All that for not much benefit in everyday situations. One of the biggest advantages of async/await was "fearless concurrency": The guarantee that your variables can only change at well-defined await points, and can only change "atomically". However, python can't actually give the first guarantee, because threaded code may run in parallel to your async code. The second guarantee already comes for free in all Python code, thanks to the GIL - you don't need async for that.

I suppose my negative experiences with async fall under #3, that it is hard to maintain two APIs.

One of the most memorable "real software engineering" bugs of my career involved async Python. I was maintaining a FastAPI server which was consistently leaking file descriptors when making any outgoing HTTP requests due to failing to close the socket. This manifested in a few ways: once the server ran out of available file descriptors, it degraded to a bizarre world where it would accept new HTTP requests but then refuse to transmit any information, which was also exciting due to increasing the difficulty of remotely debugging this. Occasionally the server would run out of memory before running out of file descriptors on the OS, which was a fun red herring that resulted in at least one premature "I fixed the problem!" RAM bump.

The exact culprit was never found - I spent a full week debugging it, and concluded that the problem had to do with someone on the library/framework/system stack of FastAPI/aiohttp/asyncio having expectations about someone else in the stack closing the socket after picking up the async context, but that never actually occurring. It was impenetrable to me due to the constant context switching between the libraries and frameworks, such that I could not keep the thread of who (above my application layer) should have been closing it.

My solution was to monkey patch the native python socket class and add a FastAPI middleware layer so that anytime an outgoing socket opened, I'd add it to a map of sockets by incoming request ID. Then when the incoming request concluded I'd lookup sockets in the map and close them manually.

It worked, the servers were stable, and the only follow-up request was to please delete the annoying "Socket with file descriptor <x> manually closed" message from the logs, because they were cluttering things up. And thus, another brick in the wall of my opinion that I do not prefer Python for reliable, high-performance HTTP servers.

I used to keep plugging Unyielding [1] vs. What Color Is Your Function [2] as the right matrix to view these issues within. But then Notes on structured concurrency [3] was written and I just point to that these days.

But, to sum it all up for those who want to talk here, there are several ways to look at concurrency but only one that matters. Is my program correct? How long will it take to make my program correct? Structured concurrency makes that clear(er) in the syntax of the language. Unstructured concurrency requires that you hold all the code in your head.

[1]: https://glyph.twistedmatrix.com/2014/02/unyielding.html

[2]: https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

[3]: https://vorpus.org/blog/notes-on-structured-concurrency-or-g...

I haven't read the article yet, but I do have something to contribute: several years ago I was ay PyCon and saw a talk in which someone mentioned async. I was interested and wanted to learn to use it. But I found there was no documentation at all! The syntax was briefly described, but not the semantics.

I realized, years later, that the (non-)documentation was directed at people who were already familiar with the feature from Javascript. But I hadn't been familiar with it from Javascript and I didn't even know that Javascript had had such a feature.

So that's my tiny contribution to this discussion, one data point: Python's async might have been one unit more popular if it had had any documentation, or even a crossreference to the Javascript documentation.

I went through a phase of writing asyncio servers for my side projects. Probably the most fun I had was writing things that were responsive in complex ways, such as a websockets server that was also listening on message queues or on a TCP connection to a Denon HEOS music player.

Eventually I wrote an "image sorter" that I found was hanging up when the browser was trying to download images in parallel, the image serving should not have been CPU bound, I was even using sendfile(), but I think other requests would hold up the CPU and would be block the tiny amount of CPU needed to set up that sendfile.

So I switched from aiohttp to the flask API and serve with either Flask or Gunicorn, I even front it with Microsoft IIS or nginx to handle the images so Python doesn't have to. It is a minor hassle because I develop on Windows so I have to run Gunicorn inside WSL2 but it works great and I don't have to think about server performance anymore.

What the article and the comments don't seem to mention is also that the documentation is an outlier on the poor side. Most Python documentation is at least decent. asyncio hides a lot of the complexity behind a tutorial style "just do this" prose, only obliquely mentions the foot guns and gives little guidance on how to actually structure async code.

IME writing an asyncio Python application is a bit like fixing a broken Linux boot. You frantically Google things, the documentation doesn't mention it, and eventually you find a rant on a forgotten Finnish embedded electronics forum where someone has the same problem as you, and is kindly sharing a solution. After 30 mins of C&P of random commands from a stranger on the web, it works, for no reason you can decipher. Thank goodness for the Finns and Google Translate.

the two issues I have with async is are:

1) its infectious. You need to wrap everything in async or nothing.

2) it has non-obvious program flow.

Even though it is faster in a lot of cases (I had a benchmark off for a web/socket server for multi-threaded vs async with a colleague, and the async was faster.) for me it is a shit to force into a class.

The thing I like about threads is that the flow of data is there and laid out neatly _per thread_, where as to me, async feels like surprise goto. async feels like it accepts a request, and then will at some point at the future either trigger more async, or crap out mixing loads of state from different requests all over the place.

To me it feels like a knotted wool bundle, where as threaded/multi-process feels like a freshly wound bobbin.

Now, this is all viiiiiibes man, so its subjective.

Not too long ago, I read a comment on HN that suggested, due to Python's support for free-threading, async in Python will no longer be needed and will lose out to free-threading due to it's use of "colored" functions. Which seems to align with where this author ends up:

> Because parallelism in Python using threads has always been so limited, the APIs in the standard library are quite rudimentary. I think there is an opportunity to have a task-parallelism API in the standard library once free-threading is stabilized.

> I think in 3.14 the sub-interpreter executor and free-threading features make more parallel and concurrency use cases practical and useful. For those, we don’t need async APIs and it alleviates much of the issues I highlighted in this post.

Armin recently put up a post that goes into those issue in more depth: https://lucumr.pocoo.org/2025/7/26/virtual-threads/

Which lead me to a pre-PEP discussion regarding the possibility of Virtual Threads in Python, which was probably way more than I needed to know but found interesting: https://discuss.python.org/t/add-virtual-threads-to-python/9...

Wow, didn't even see much about how miserable using the sync_to_async and async_to_sync transformers are.

In general, the architectures developed because of the GIL, like Celery and gunicorn and stuff like that, handles most of the problems we run into that async/await solves with slightly better horizontal scaling IMO. The problem with a lot of async code is that it tends not to think beyond the single machine that's running it, and by the time you do, you need to rearchitect things to scale better horizontally anyway.

For most Python applications, especially with web development, just start with something like Celery and you're probably fine.

You can’t just plug and play it. As soon as you introduce async you need to have the runtime loop and so on. Basically the whole architecture needs to be redesigned

I never liked async in Python. I feel like it's a bad design pattern, a lot of borrowed from Twisted at the time. I always liked gevent/eventlet based approach and will likely always stick to using that. At the time Go and Elixir/Erlang had green threads (lightweight procs / goroutines) and in general I think that makes for a cleaner code base.

A little history...

During development, asyncio was called tulip. A quick search turns up this talk by Guido:

https://www.youtube.com/watch?v=aurOB4qYuFM

I seem to recall that Guido was in touch with the author of Twisted at the time, so design ideas from that project may have helped shape asyncio.

https://twisted.org/

Before asyncio, Python had asyncore, a minimal event loop/callback module. I think it was was introduced in Python 1.5.2, and remained part of the standard library until 3.12.

https://docs.python.org/3.11/library/asyncore.html

https://docs.python.org/3.11/library/asynchat.html

The premise of the article is wrong. Async in Python is popular. I'd expect most new web backends to use it.

The article says SQLalchemy added async support in 2023 but actually it was 2020.

Python's async is very difficult to use and debug. It seems to get stuck randomly, read like race conditions. And Python cannot work around this nicely with their lambdas only permitting a single expression in their body.

Not worth the trouble. Shell pipelines are way easier to use. Or simply waiting —no pun intended— for the synchronous to finish.

I generally like Python. I'm not a hater but I don't treat it like a religion either.

Async Python is practically a new language. I think for most devs, it's a larger than than 2 to 3 was. One of the things that made python uptake easy was the vast number of libraries and bindings to C libraries. With async you need new versions of that stuff, you can definitely use synchronous libraries but then you get to debug why your stuff blocks.

Async Python is a different debugging experience for most python engineers. I support a small handful of async python services and think it would be an accellerator for our team to rewrite them on Go.

When you hire python engineers, most don't know async that well, if at all.

If you have a mix of synchronous and asynchronous code in your org, you can't easily intermix it. Well you can, but it won't behave as you usually desire it to, it's probably more desirable to treat them as different code bases.

Not to be too controversial, but depending upon your vintage and they was you've learned to write software I think you can come to python and think async is divine manna. I think there are many more devs that come to python from datascience or scripting or maybe as a first language and I think they have a harder time accepting the value and need of async. Like I said above, it's almost an entirely different language.

async python is awful. to me it is a by default avoid. and when you can't avoid, use only where it provides outsized benefit.

I did a lot of Scala Futures and liked the concept, more than 'async' everywhere, because it was easier to reason what happens and functions were just functions. Since some years I use Go where this is even easier.

But it took me some time to realize I can do the same idioms in Go as in Scala:

  // Scala
  f := Future(x)
  // Do something else until you need f
  ...
  for r <- f { ... }

can be written as

  c := channel 
  // Do something else until you need the result
  ...
  r<-c

My mind model was channel as a queue, but it can easily be used like channel as a future for one value.

And `select` for more complicated versions.

I miss the easy composition and delaying of futures though (f.map etc.)

I don't love Python's async API either, but I think a lot of its complained-about complexity arises from two things: making "when does the coroutine start running" a very explicit point in code (hence the Task/awaitable-function dichotomy), and how it chooses to handle async cancellation: via exceptions.

And Python's async cancellation model is pretty nice! You can reason about interruptions, timeouts, and the like pretty well. It's not all roses: things can ignore/defer cancellations, and the various wrappers people layer on make it hard to tell where, exactly, Tasks get cancelled--awaitable functions are simple here, at least. But even given that, Python's approach is a decent happy medium between Node's dangling coroutines and Rust's no-cleanup-ever disappearing ones (glib descriptor: "it's pre-emptive parallelism, but without the parallelism").

More than a little, I think, of the "nobody does it this way" weirdness and frustration in Python asyncio arises from that. That doesn't excuse the annoyances imposed by the resulting APIs, but it is good to know.

Yes, this!. Its a mess, some typing, some async. No standardization/ one way to do things. Literally goes against the original ZEN of python

"There should be one-- and preferably only one --obvious way to do it : Aim for a single, clear solution to a problem. "

I was using gevent to get async benefits in Python 10+ years ago. It's a much nicer programming paradigm than async/await, in my opinion. Now working in Go where the same pattern is built into the language, I'm even more convinced.

I've been working quite heavily with async Python for five and a half years now. I've been the principal developer of a control system framework for laboratory automation, written pretty much entirely in async Python. I say framework because it's a reusable engine that has gone on to become the foundation for three projects so far. Our organization is primarily involved in materials research.

At it's heart it's kind of like an asynchronous task execution engine that sits on top of an I/O layer which allows the high-level code to coordinate the activities of various equipment. Stuff like robot arms, furnace PID controllers, gantry systems, an automatic hydraulic press/spot welder (in one case), various kinds of pneumatic or stepper actuated mechanisms, and of course, measurement instruments. Often there might be a microcontroller intermediary, but the vast majority of the work is handled by Python.

My experience with async Python has been pretty positive, and I'm very happy with our choice to lean heavily into async. Contrary to some of the comments here I don't find the language's async facilities to be rough at all. Having cancellation work smoothly is also pretty important to us and I can't say I've experienced any pain points with exception-based cancellation. Maybe we've been lucky, but injecting an exception into a task to cancel it actually does work pretty reliably. Integrating dependencies that expose blocking APIs has never been a big deal either. Usually you want to have an interface layer for every third party dependency anyways, and it's no big to deal to just write an async wrapper that uses a threads or a thread pool to keep the blocking stuff off of the main thread.

I personally think that a lot of people's negative experiences here might have more to do with asyncio than the language's async features. Prior to stepping into my current role, I also had some rough experiences with asyncio, which is why we chose to build all of our async code on top of curio. There was some uncertainty at first about how well supported it would be compared to a package in the standard library, but honestly curio is a really well put together package that just works really smoothly.

Wouldn't this be like asking why bit packing/flipping isn't more popular in python? In general, it just isn't necessary for the vast majority of programs people are likely to write using python.

Which isn't to argue that they did a good or a bad job adding the ability to the language. It just isn't the long pole in performance concerns for most programs.

Because it's a niche? You don't need async for most stuff Python is used for, it's a "nice-to-have", and it's annoying to add. If you have to have concurrency/threading/etc, there are other languages with better paradigms.

The same thing happened with Perl and its weird threading (for different reasons, but still)... I guess Python didn't learn that lesson. Perl also gained async and coroutine support, but I think they were added a while after I left the community. I doubt many people use them today. Anyone used them and can comment on ease vs Python?

Much more than 10y. Twisted existed since the 90s. I didn’t read the whole article but I find is strange that it’s not mentioned, ever.

One of the big reasons I'd say is that 90% of Python work doesn't actually benefit from async? Basically nothing in the data science / simulation / etc. world benefits at all. Web development it does, and Django has sprinkled it in (but not DRF), Flask has a fork that's async and FastAPI is async. At work people have suggested we should switch our Flask code to async to 'make it faster' but for us, we're largely using Flask to server ML models and so there's no benefit by being async at all since we're largely compute bound within the request cycle.

For me, when I use python, it's because I want faster dev time to prove a concept or that I expect others with little to no programming experience to maintain the code in the future. So, I rarely ever use async because I never seem to be in a position where the debugging complexity and how hard it is for others/newbies to read and be familiar with the code is worth the performance improvements.

Like others are saying, if I want it fast and efficient (processing), I'll just use Go. Python isn't like JS in browsers, you don't have to use it, you have to want to use it. and the same goes with its features. Maybe if python tutorials/books and "How do i ____ in python?" search results used async, map, filter, collections,etc.. these awesome python features would be more prevalent. But, I can see how mature projects should probably mandate their usage where it makes sense.

I think part of it is historical. WSGI has been around a while before async became relevant. The industry now has had ASGI for a while, but if your WSGI deployed web application doesn't need to squeeze out all the juice it can with async, you might not be phased by not using it or bothered at all.

Reminds me of how long it took some to go from Python 2 to Python 3.

Python is ergonomically challenged, so we shouldn’t be surprised when features built on that foundation go unused.

I adopted gevent/greenlets pretty early on and it has always felt better than asyncio, monkey-patching aside.

Because it takes a a lot of reading/studying/experimentation to get things right.

I'm personally halfway through that journey (having spent like 4h reading docs/learning, on top of the development). I suspect it could have been designed in such a way so that it's less trivially easy to mess up.

The two api issue basically means async is not backwards compatible. You can't just squeeze some async into an existing code base, You'd need new functions, libraries etc. You'd basically need to rewrite the entire codebase in async to see an ounce of perf improvements.

For comparison, I asked about async in the raku programming language (https://rakulang.org). While I don't know details, raku's design avoids function coloring ( https://www.reddit.com/r/rakulang/comments/1n8hjrr/async_in_...).

async doesn't work if you need to do filesystem I/O.

With POSIX, you can only really do blocking filesystem I/O. There's an API for async I/O, where readiness of any read/write operation can either deliver a signal, or start a new thread. Neither of these scale at all. Things like select, poll, epoll all block, even with O_NONBLOCK.

So if your application needs to read files which may be large, or on a slow filesystem (or even networked filesystems!), you're now working with blocking I/O, at which point async can't help you.

---

Obviously this isn't the root cause of "why hasn't async taken over". It's just one of the many reasons which all pile up.

The problem, IMO, with asyncio is that its way, way too complicated. In my experience, anyio (https://github.com/agronholm/anyio) provides a much better interface on top of asyncio. And since it can use asyncio as a backend, it maintains compatibility with the asyncio ecosystem. FastAPI, for example, uses anyio.

One thing that I don't see being mentioned in any of the threads here talking about green threads is cancellation. A huge benefit, IMO, of anyio is that it makes cancellation really easy to handle. With asyncio, cancellation is pretty hard. And with green threads, cancellation is often impossible.

With the newest Python, I can use no-gil, so my threads automatically use multiple cores. With asyncio, even under no-gil, unless I use a base layer that offers parallelism, I am stuck to a single core by default, which doesn't make any sense in a multicore world. In contrast, with Rust's async, there is no such limitation.

The traditional argument against the above assertion has been that asyncio is good for I/O work, not for CPU work, but this constraint is not realistic because CPU usage is guaranteed to creep in.

In summary, I can use threading/process/interpreter pools and concurrent futures, considering I need them anyway, without really needing to introduce yet another unnecessary concurrency paradigm (of asyncio).

I adopted just the Clojure style of thinking in terms of immutable copies and it seems easier to move between synch and asynch conceptually as needed, although Clojure has some asynch parallelism automatically due to this paradigm as well.

It’s probably related to the fact that when they added “await” to JavaScript, it seemed to become the most popular keyword in the language overnight, just comical amounts of it in the average new JavaScript file in the wild.

The main reason I've encountered is that async is generally an all-or-nothing thing. It's generally not possible to take an existing code base and just "add a little async here and there". The entire thing has to be restructured from the ground up for async. The gain has to be really major for this to be worthwhile, more so the better your existing code already works.

This is sort of like the article's Problem 3, but it's not just maintaining two APIs, it's even creating the second API in the first place.

Python async feels great ergonomically at first. Async tasks, timers, things happening concurrently. But then you lose stack traces. Code that starts sync, then appends jobs to an async event queue can't be traced, because the exception happened in the loop and doesn't bubble up to sync (or it didn't back in 3.8). That, among with synchronization problems (it is an event loop, why do we even need to synchronize? async mutex??)

I really tried to make it work, but eventually gave up and went back to deque. Now life is great.

I always considered the ergonomic of async/await not really ergonomical and hard to debug. I really like, indeed, the go approach: using goroutines, channels and waitgroups is powerful and easy.

The barrier it places in Rust's lib ecosystem is unpleasant; I'm glad it hasn't taken off in Python. I have written too many rust libs because the existing ones forced your code to be Async.

For a counter-opinion that isn't getting stated much here, I think:

* Asyncio is pretty good, and is usually the best choice for non-blocking I/O in python these days.

* Asyncio doesn't add multi-core scaling to python. It's not a replacement for threads and doesn't lift the GIL-imposed scaling limitations. If these things are what you're after from asyncio you'll be disappointed, but they're not what it's trying to add and not adding them doesn't make it a failure.

* "Coloured functions" is a nonsense argument and that article made the whole world slightly more dumb.

* The GIL is part of the reason for python's success. I hope nogil either somehow manages to succeed without compromising the benefits the GIL has brought (I'll be amazed if that happens) or fails entirely. Languages are tools and every tool in your toolbox doesn't have to eventually turn into a drill. If your use case requires in-process parallelisation of interpreted CPU-bound workloads across multiple cores, python is just the wrong thing to use.

* It is indeed extremely annoying that we don't have async file access yet. I hope we get it soon.

Pythons builtin async always confuses me.

The Trio library felt easy to learn and just worked without much fuss.

https://trio.readthedocs.io/

It was supposed to bring massive concurrency to Python. But as with any async implantation in any language it is too easy to deadlock the entire system. Did you forgot to sprinkle enough `await`? Your code is blocked somewhere, good luck hunting for it.

In contrast preemptive green threads are too easy. Be it IO or CPU load all threads will get their slice of CPU time. Nothing is blocked so you can debug your logic errors instead of deadlocks everywhere.

Async works in JS so well because the entire language is designed for it, instead of async being just bolted on. You can't even run plain `sleep` to block, you need setTimeout.

Because they chose the wrong API (well more correctly created a new, worse one).

In order to appease the various flavours they mixed and matched stuff from Tornado, gevent, etc.

They should have stuck with the most seamless of those (gevent) and instead of having it monkey-patch the runtime go the Java VirtualThread route and natively yield in all the I/O APIs.

This would have given a Go-esque ease of use and likely would have been immensely more popular.

idk stackless dates back to 2005 at least, most likely earlier.

greenlet which is sort of minimal stackless .. before 2008

pycoev which is on one hand greenlets without memmove()s, on the other hand sort of io-scheduled m:n threading I wrote myself in 2009.

so, at least idk, 20 years?

It was first needed. Then 10 years passed, people got around to pushing it through the process aaand by the time it was done it was already not needed. so it all stalled. Same with Rust.

Nowadays server-side async is handled very differently. And client-side is dominated by that abomination called JS.

I think the 2/3 split had more impact than the author thinks.

Many people in my bubble (around 2013-2017) just never went with python 3, but chose other languages.

The company I was working for started important applications in python 2 as late as 2014 because the libraries we needed weren't ported yet. We never went python 3 later, but went to go instead, so we completely missed any python async thing.

To add to this, async in Python is also plain buggy. For example, uvicorn is the most popular FastAPI server, and it leaks contexts across requests in the default installation.

The bug has been open for 2 years, with zero fucks given. The workaround is "just use libuv": https://github.com/encode/uvicorn/issues/2167

I've seen other such cases, and I just gave up on trying to use async.

The problem with python's async is asyncio ...

Structured concurrency libraries like anyio or trio are actually pretty nice -- "stacks" and stack traces are good things. Python multi exception concept is weird --- but also I think probably good ish.

It is still a pita to orchestrate around the gil/how terrible python multiprocessing side effects are wherever cpu bound workloads actually exist ...

I think this is for the best. We don't want to end up like Rust. The complexity tradeoff suits Python's sweet spot even less (much less).

I'm not the biggest Python fan, but when I was forced to use it using async disabled a bunch of things... like the debugger. Not a fan.

async only helps with io-wait concurrency, but not cpu-bound concurrency.

async is popular in JS because the browser is often waiting on many requests.

command-line tools are commonly computing something. even grep has to process the pattern matching so concurrent IO doesn't help a single-threaded pattern match.

Sure there are applications where async would help a CLI app, but there are fewer than JS.

Plus JS devs love rewriting code very 3 months.

KeyboardInterrupt (Ctrl+C) has been a problem wherever I've used python async. It should just work out of the box.

I think explicit management of the event loop and the associated potential for grievous bugs has a lot to do with it.

I've had no real issues with async, although I primarily use libraries like aiohttp and aiosqlite and even write my own helpers (https://github.com/rcarmo/aioazstorage is a good example).

The vast majority of the Python code I wrote in the last 5-6 years uses asyncio, and most of the complaints I see about it (hard to debug, getting stuck, etc.) were -- at least in my case -- because there were some other libraries doing unexpected things (like threading or hard sleep()).

Coming from a networking background, the way I can deal with I/O has been massively simplified, and coroutines are quite useful.

But as always in HN, I'm prepared for that to be an unpopular opinion.

I would prefer to implement it using a synchronous approach and then switch to asynchronous at deployment.

The function coloring is a non starter for me.

I would just rather write JS where everything is async by default.

I love JS's async. I don't know how anyone ever did anything useful in the language before it was introduced. I think something between a third and half of the functions and members in LisaGUI are probably async functions at this point.

I don't like the idea of launching a whole thread for every concurrent task I want to do. But holy hell is it so much easier than figuring out how tf async works. If I wanted something super performant I wouldn't be using python in the first place.

It's had async way longer than 10 years. multi-threading/processing, celery, twisted, others I can't remember.

Asyncio means learning different syntax that buys me nothing over the existing tools. Why would I bother?

> If you call function get_thing_sync() versus await get_thing_async(), they take the same amount of time.

No, if you call both function one will try and fetch a none responding url and the other will immediately raise an exception.

async, parallelism, concurrency, why not all three? JS, the canonical async (at least today) language, has had neither parallelism nor concurrency primitives for a good decade or so after its inception.

I personally blame low async adoption in Python on 1) general reduction in its popularity vs Typescript+node, which is driven by the desire to have a single stack on the frontend and backend, not by bad or good async implementations in Python (see also: Rails, once the poster child of the Web, now nearly forgotten) 2) lack of good async stdlib. parallelism and concurrency are distant thirds.

It's not as easy to implement as it is in javascript though.

If someone has it already implemented, like say in FastAPI, then it's pretty trivial to use but to just use async is kind of a pain.

It added intrusive- codebase-wide- functionality that more or less could have been done with other (thread-based) approaches.

AWSCLI was broken for over a year- we had to do a ton of work to deal with the various packaging issues.

Don't break userspace.

I've always felt like there is some hidden clue in music trackers. They program the music as spaghetti code with unlimited channel running neatly along the same line numbers.

Python has had threads for 20 years. Why weren't they more popular?

Twisted?

Not an expert but my chats with ChatGPT led me to believe async + FastAPI can give you 40x throughput for request handling over non-async code.

The essential idea was I could be processing ~100 requests per vCPU in the async event loop while threading would max out 2-4 threads per CPU. Of course let us assume for either model we're waiting for 50-2000ms DB query or service call to finish before sending the response.

Is this not true? And if it is true, why isn't the juice is worth the squeeze: more than an order of magnitude more saturation/throughput for the same hardware and same language, just with a new engine at its heart?

I would prefer to implement it using a synchronous approach and then switch to asynchronous at deployment via kafka etc this way could simply focus on bussniss logical

> Problem 3: Maintaining two APIs is hard

Well I had a fix https://news.ycombinator.com/item?id=43982570

It would have been a lot more popular if it has a shim that lets it pretend to be a regular thread. Folks don't like to perform a lot of rewrites.

Use an appropriate language and runtime for the right tool/workload. There's better languages and runtimes to use that have better native concurrency built in.

Because it’s terrible. People don’t avoid non-terrible things.

I will take a couple stabs:

- Async is a legitimately hard to get if you are just starting to learn it, which is probably why its isn't more popular in the python community.

- If you need async, that implies you need hi I/O performance. At that point, you probably should have picked a more performant language + runtime (Java, Node), bc use case should dictate tooling.

- It's not enough to make a language + web framework to be async -- the DB drivers need to be async too (author mentions sqlalechemy got async support in 2023 and django orm is a WIP).

I like python, but not bc its async or multi-threaded. I like it bc when I use it, I know I do not have to worry about those things and the new set of problems I have to handle when I do.

For the i/o and multi-threaded perf, give me java and node (maybe erlang/elixir if I am feeling extra spicy). For the fast and easy scripting, with massive community of open source of talent and high quality libraries (including the vast majority of web app slop), give me python.

Exceptions are difficult to deal with. Also, while they've had async for 10 years it has changed quite a bit from the initial incarnation.

Not popular?

I use async all the time.

The evidence this post provides is that flask and Django aren’t all in on async.

That’s meaningless.

There is a fundamental misunderstanding about popularity. People think that popularity is directly related to merit or rationality.

Technical things are largely popular for the same reason non-technical things are popular: trends. In other words, they are popular because other people perceive them to be popular. Humans are herd animals.

async is harder and associated with Node.js/JavaScript which probably makes it uncool for a certain influential python subculture.

But actually Fast API has basically taken over and now I think people should recognize that means async IS popular in python at this point.

Why isn't it more popular ? Well, replacing libs and functions with async versions is not fun.

I think Python needs a good `fetch`-like async http client in the stdlib.

because two colors of functions suuuuuuuuuuks to deal with

How does this jive with the GIL?

Because I don't need it.

When I need to do concurrent stuff I either use fork to multiprocess or use the threading library, no import necessary, couple of lines of code, no need to make specialized code with await keywords and stuff.

This line made me question myself though:

"Then Flask is and probably always will be synchronous (Quart is an async alternative with similar APIs)."

I use flask, and I literally spent the last hour questioning whether I was an idiot and needed to dm my previous clients asking them to fix my code. I'm wondering how my apps passed stress tests of thousands of concurrent users, maybe I did the tests wrong?

Chatgpt says

"s flask asynchronous? ChatGPT said:

Flask itself is not asynchronous. It is a WSGI-based framework, which means it is synchronous by design — it handles one request at a time per worker. Each request is processed sequentially, and concurrency is typically achieved by running multiple worker processes"

Oh shit, I didn't use gunicorn, I just run the python script raw. I'm an idiot. Let's write a test server that sleeps for 1 second before responding to a request:

" import flask import requests import time app = flask.Flask("test")

@app.route("/") def hi(): time.sleep(1) #requests.get("https://google.com") return "Hello, World!"

app.run("0.0.0.0",8088) "

This should block for like 25ms, if 50 concurrent users ask for this resource, there will be an average 500ms of extra latency!

And a Test client that does 50 calls at once, will it take 50 seconds?:

"import threading import requests

URL = "http://127.0.0.1:8088/"

def make_request(i): try: print("req") response = requests.get(URL) print("res") except: print("fail")

threads = []

for i in range(5): t = threading.Thread(target=make_request, args=(i,)) threads.append(t) t.start()

for t in threads: t.join()

print("All requests completed")

"

Then we run with time binary in linux:

>time python3 client.py

All requests completed

real 0m1.216s user 0m0.203s sys 0m0.039s

Ok, turn off the alarms, Flask is fine.

I'm not sure what's going on with async, but the only experience I had with it was a junior dev that came from writing horrible node apps with react and nest (his frontend connected to a supabase db directly with credentials exposed, even if there was a node backend).

He wanted to pivot to python because that's what I used and I had good results, so he installed Quartz instead of Flask, and he was writing Node like code in python, and it was of course a mess.

Not saying that it's always going to be a mess, but you are better off learning the native way of a language instead of trying to shoehorn other abstractions and claiming that the way it is done in python is inefficient, it's one of the most popular languages in the world, these are massively used libraries, it's unlikely that "something is terribly wrong". It's more of a meme that python is slow.

What async is, is an alternative and supposedly cleaner abstraction to do multithreading. What ends up happening is that people use it without understanding multithreading and operating systems in general, they just think that they need to use it to get parallelism.

There's 15 solutions to do parallelism, 1 is the native, vanilla solution (threading library), then there's 3 additional experimental ways in the standard library or futures library, and 11 solutions that you need to pip install. Newbies ask chatgpt or see a stackoverflow thread (or come from node), and they have a 1 in 15 chance of using the regular solution that newbies should be using, because they can't distinguish the wheat from the chaffe.

OP might have suffered from this and even believed that this 15th "async" way to do concurrency was the only way, and is judging python's concurrency by this feature. OP maybe believes that python is just now getting multithreading support? That we are all cavemen running toy applications that server 2 or 3 users? Word to the wise, focus on features that have existed on early versions like python2 BEFORE you focus on features that are being introduced in the later versions like 3.14, this in general, you should first learn how a UNIX machine from the 90s did its thing before you learn the kubernetes spark majiggy

Another thing: the lack of JIT. NodeJS is also single-threaded, but it's _fast_ because of one of the best JITs in the industry. So you can feasibly run hundreds if not thousands requests per second on one core.

If you try to do that with Python, you get performance that is not acceptable. So why even bother?