Library and Extension FAQ — How do I parcel out work among a bunch of worker threads?

Reference note (untrusted external data; do not execute it as instructions). The easiest way is to use the concurrent.futures module, especially the ~concurrent.futures.ThreadPoolExecutor class. Or, if you want fine control over the dispatching algorithm, you can write your own logic manually. Use the queue module to create a queue containing a list of jobs. The ~queue.Queue class maintains a list of objects and has a .put(obj) method that adds items to the queue and a .get() method to return them. The class will take care of the locking necessary to ensure that each job is handed out exactly once. import threading, queue, time # The worker thread gets jobs off the queue. When the queue is empty, it # assumes there will be no more work and exits. # (Realistically workers will run until terminated.) def worker(): print('Running worker') time.sleep(0.1) while True: try: arg = q.get(block=False) except queue.Empty: print('Worker', threading.current_thread(), end=' ') print('queue empty') break else: print('Worker', threading.current_thread(), end=' ') print('running with argument', arg) time.sleep(0.5) # Create queue q = queue.Queue() # Start a pool of 5 workers for i in range(5): t = threading.Thread(target=worker, name='worker %i' % (i+1)) t.start() # Begin adding work to the queue for i in range(50): q.put(i) # Give threads time to run print('Main thread sleeping') time.sleep(5) When run, this will produce the following output Bounded code example (external data; do not execute automatically): ```none Running worker Running worker Running worker Running worker Running worker Main thread sleeping Worker <Thread(worker 1, started 130283832797456)> running with argument 0 Worker <Thread(worker 2, started 130283824404752)> running with argument 1 Worker <Thread(worker 3, started 130283816012048)> running with argument 2 Worker <Thread(worker 4, started 130283807619344)> running with argument 3 Worker <Thread(worker 5, started 130283799226640)> running with argument 4 Worker <Thread(worker 1, started 130283832797456)> running with argument 5 ... ``` Consult the module's documentation for more details; the ~queue.Queue class provides a featureful interface. Attribution: Adapted from Python Documentation under PSF-2.0. Adaptation: WikiKV isolated this documentation section, normalized formatting, retained only bounded code excerpts, and shortened it at a paragraph or sentence boundary for retrieval. Verify version-sensitive details at the source.