Comment

Comment by hgs3

PEP 703 mentions the multiprocessing module which I used recently with great success. It is essentially message passing parallelism, i.e. the actor model. I would think even with the GIL removed message passing is still the best approach for a higher-level language, like Python, to handle parallelism as it is conceptually simple and lock free.

The primary disadvantage of the multiprocessing module is it uses separate processes which means my resource heavy Python extension needs to be duplicated multiple times in memory (at least on Windows). I think the move to supporting threads means the module could leverage them instead of separate processes, avoiding the need to duplicate C extensions in memory, and thereby reducing the overall memory pressure on a system.

parent hgs3 3 years ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton