Concurrency and parallelism are similar terms, but they are not the same thing. This post looks at how to speed up CPU-bound and IO-bound operations with multiprocessing, threading, and AsyncIO. Older article by Amal Shaji.
Concurrency is the ability to run multiple tasks on the CPU at the same time. Tasks can start, run, and complete in overlapping time periods. In the case of a single CPU, multiple tasks are run with the help of context switching, where the state of a process is stored so that it can be called and executed later.
Parallelism, meanwhile, is the ability to run multiple tasks at the same time across multiple CPU cores.
The article also mentions:
- IO-bound operation
- Sync example
- Threading example
- concurrent.futures example
- AsyncIO example
- CPU-bound operation
- Sync example
- Multiprocessing example
- concurrent.futures example
It’s worth noting that using multiprocessing to execute the make_request function will be much slower than the threading flavor since the processes will be need to wait for the IO. The multiprocessing approach will be faster then the sync approach, though.
Similarly, using concurrency for CPU-bound tasks is not worth the effort when compared to parallelism.
That being said, using concurrency or parallelism to execute your scripts adds complexity. Your code will generally be harder to read, test, and debug, so only use them when necessary for long-running scripts. All of the code examples in this post can be found in the parallel-concurrent-examples-python repo. Good read!
[Read More]