Comment

Comment by BiteCode_dev

Don't follow this blog post advice.

Manually dealing with threads and processes is useful if you want to build a framework or a very complex workflow. But chances are you just want to run stuff concurrently, in the background.

In that case (which is most people case), you really want to use one of the stdlib pools: it takes care of sync, serialization, communication with queues, worker life cycle, task distribution, etc. for you.

Plus in such case, waiting for results is super easy:

    import time
    import random

    from concurrent.futures import ThreadPoolExecutor, as_completed

    # the work to distribute
    def hello():
        seconds = random.randint(0, 5)
        print(f'Hi {seconds}s')
        time.sleep(seconds)
        print(f'Bye {seconds}s')
        return seconds

    # max concurrency is 2
    executor = ThreadPoolExecutor(max_workers=2)
   
    # submit the work
    a = executor.submit(hello)
    b = executor.submit(hello)

    # and here we wait for results
    for future in as_completed((a, b)):
        print(future.result())
Want multiple process instead ? It's the same API:

    import time
    import random

    from concurrent.futures import ProcessPoolExecutor, as_completed

    def hello():
        seconds = random.randint(0, 5)
        print(f'Hi {seconds}s')
        time.sleep(seconds)
        print(f'Bye {seconds}s')
        return seconds

    # Don't forget this for processes, or you'll get in trouble
    if __name__ == "__main__":

        executor = ProcessPoolExecutor(max_workers=2)

        a = executor.submit(hello)
        b = executor.submit(hello)

        for future in as_completed((a, b)):
            print(future.result())
This is Python. Don't make your life harder that it needs to be.

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton