Speed up I/O bound programs with threading

Here follows what I wanted to highlight from the following article. Please give huge props to Jim Anderson for his wonderful work.

_{Anderson Jim, (2021), Speed Up Your Python Program With Concurrency. Real Python.}

A common I/O Bound Program

Downloading content over the network is a common I/O bound problem. For our example, you will be downloading web pages from a few sites.

Synchronous Version

import requests
import time


def download_site(url, session):
    with session.get(url) as response:
        print(f"Read {len(response.content)} from {url}")


def download_all_sites(sites):
    with requests.Session() as session:
        for url in sites:
            download_site(url, session)


if __name__ == "__main__":
    sites = [
        "https://www.jython.org",
        "http://olympus.realpython.org/dice",
    ] * 80
    start_time = time.time()
    download_all_sites(sites)
    duration = time.time() - start_time
    print(f"Downloaded {len(sites)} in {duration} seconds")

An important note: Jim uses a Session from requests. Creating a Session object allows requests to do some fancy networking tricks and really speed things up. On my own local testing of the program running time, it went from 19 seconds without a session and 9 seconds !

Sync Code Benefits

The great thing about this version of code is that, well, it’s easy. It was comparatively easy to write and debug. It’s also more straight-forward to think about.

Sync Code Disadvantages

The big problem here is that it’s relatively slow compared to the other solutions we’ll provide. This example took from 9 seconds to up 19 seconds to run.

Threading Version

A thread is a separate flow of execution. This means that your program will have two things happening at once.

Here is Jim same program using threading:

import concurrent.futures
import requests
import threading
import time

thread_local = threading.local()

def get_session():
    if not hasattr(thread_local, "session"):
        thread_local.session = requests.Session()
    return thread_local.session


def download_site(url):
    session = get_session()
    with session.get(url) as response:
        print(f"Read {len(response.content)} from {url}")


def download_all_sites(sites):
    with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
        executor.map(download_site, sites)


if __name__ == "__main__":
    sites = [
        "https://www.jython.org",
        "http://olympus.realpython.org/dice",
    ] * 80
    start_time = time.time()
    download_all_sites(sites)
    duration = time.time() - start_time
    print(f"Downloaded {len(sites)} in {duration} seconds")

One of the main difficulties with threading is because the operating system is in control of when your task gets interrupted and another task starts, any data that is shared between the threads needs to be protected, or thread-safe. Unfortunately requests.Session() is not thread-safe. As such, each thread needs to create its own requests.Session() object.

Threading Code Benefits

This concurrent version took only 3 seconds to run !

It uses multiple threads to have multiple open requests out to web sites at the same time, allowing your program to overlap the waiting times and get the final result faster!

Threading Code Disadavantages

The code is more complex and you really have to give some thought to what data is shared between threads.