Python Threads

In the following example, we use multiple threads to run heavy, again 80 times. Each invocation gets its own thread:

import threading
import time

# A CPU heavy calculation, just
# as an example. This can be
# anything you like
def heavy(n, myid):
    for x in range(1, n):
        for y in range(1, n):
            x**y
    print(myid, "is done")

def threaded(n):
    threads = []

    for i in range(n):
        t = threading.Thread(target=heavy, args=(500,i,))
        threads.append(t)
        t.start()

    for t in threads:
        t.join()

if __name__ == "__main__":
    start = time.time()
    threaded(80)
    end = time.time()
    print("Took: ", end - start)

This takes about 47s on my system. If the heavy function had a lot of blocking IO, like network calls or filesystem operations, this would be a big optimization though. The reason this is not an optimization for CPU bound functions, is the GIL!

If Python wouldn’t have the GIL, this would be much faster. But despite having 80 threads, this runs roughly as fast as the baseline. The baseline is, in fact, even a little faster because it does not have all the overhead of thread creation and switching between threads.

This is the GIL at work. Each thread takes turns, instead of running all at once. If heavy would have been an I/O bound function, however, this would have given us a tremendous speed increase. Let’s test this! We can simulate I/O bound by using time.sleep(). I modified the heavy function in the next code fragment:

import threading
import time

# An I/O intensive calculation.
# We simulate it with sleep.
def heavy(n, myid):
    time.sleep(2)
    print(myid, "is done")

def threaded(n):
    threads = []

    for i in range(n):
        t = threading.Thread(target=heavy, args=(500,i,))
        threads.append(t)
        t.start()

    for t in threads:
        t.join()

if __name__ == "__main__":
    start = time.time()0
    threaded(80)
    end = time.time()
    print("Took: ", end - start)

Even though we have 80 threads all sleeping for two seconds, this code still finishes in a little over two seconds. While sleeping, Python will schedule other threads to run. Sweet!


If you liked this page, please share it with a fellow learner: