Python) asyncio/Coroutines, Concurrency and Parallelism
This post was migrated from Tistory. You can find the original here.
Parallelism
task1 ======
task2 ======
task3 ======
Actually runs at the same time.
Concurrency
task1 === **===**
task2 === **===**
task3 === **===**
Doesn’t actually run at the same time, but multiple tasks make progress in an interleaved way.
coroutine
co(operative) + routine = a routine that runs on equal footing with the main routine, executing sequentially alongside it.
It’s the preferred way to write asyncio applications.
The difference from a subroutine is that instead of terminating once it finishes running, it can be suspended — which is why we say it has an “equal” relationship with the main routine.
Since it doesn’t run on a separate thread, there’s no context-switching cost.
asyncio
A module for asynchronous programming built on coroutines.
Because it supports suspension, it enables concurrent programming.
“Application developers should typically use the high-level asyncio functions, such as asyncio.run(), and should rarely need to reference the loop object or call its methods. This section is intended mostly for authors of lower-level code, libraries, and frameworks, who need finer control over the event loop behavior.” -documents-
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
def run(main, *, debug=None):
...
loop = events.new_event_loop()
try:
events.set_event_loop(loop)
if debug is not None:
loop.set_debug(debug)
return loop.run_until_complete(main)
finally:
try:
_cancel_all_tasks(loop)
loop.run_until_complete(loop.shutdown_asyncgens())
finally:
events.set_event_loop(None)
loop.close()
The asyncio.run() function includes both the creation and execution of the loop.
1
2
3
4
5
6
7
8
# ComfyUi
...
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
...
try:
loop.run_until_complete(...)
....
Looking at ComfyUI, it uses the loop directly instead of run(). Since the structure shares a single asyncio loop across multiple threads, it seems this is done in order to make use of the loop object in various places.
And when multiple threads share the same resource, you’d normally need to consider “thread synchronization” — the loop appears to provide methods for that.
References
https://docs.python.org/ko/3/library/asyncio-task.html#awaitables
https://docs.python.org/ko/3/library/asyncio-eventloop.html#event-loop-methods
https://brownbears.tistory.com/540
[[Python] Digging into asyncio
asyncio is used a bit differently depending on the Python version. The explanation below is written based on Python 3.8. What is asyncio? Supported since Python 3.5, asyncio is a module for asynchronous programming.
brownbears.tistory.com](https://brownbears.tistory.com/540)