🔒 Thread Pool Dependency Graph

[drifting current]

Hi, got a question about thread pool concurrency. I have a challenging problem, where I have the code:

from concurrent.futures import ThreadPoolExecutor
import time

N_THREADS = 12

pool = ThreadPoolExecutor(max_workers=N_THREADS)

def JobA():
    def work():
        time.sleep(5)
        return 1

    return pool.submit(work)

def JobB(intermediate):
    def work():
        time.sleep(3)
        return intermediate * 2

    return pool.submit(work)

def JobC():
    def work():
        time.sleep(4)
        return 1

    return pool.submit(work)

def job():
    r1 = JobA()
    r2 = JobB(r1)
    r3 = JobC()
    
    return r1 + r2 + r3 

This can be thought as a dependency graph:

A -> B -\
         -> RESULT
     C -/

How do I keep the syntax similar to what is seen in the function job while optimizing the usage of the thread pool?

[glossy mist]

@drifting current

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[twilit tulip]

U mean JobB(r1)?

Idk much about threads but this code seems fine

[drifting current]

Yeah, r1, the code isn't fine, it's just a fake example I just came up with

JobA, JobB, and JobC are all asynchronously executed, what I'd want is a way to automatically join JobA to JobB

And how to join the result to Job A through C

[twilit tulip]

Make a function that takes 2 functions and joins them?

Like

def link(f1, f2):
  def wrapper(*args, **kwargs):
    r=f1(*args, **kwargs)
    return f2(r)
  return wrapper

Or join

def join(f1,f2,f3):
  def wrapper(args1, args2):
    return f3(f1(args1), f2(args2))

@drifting current wouldnt these work?

[glossy mist]

Python help channel closed for inactivity

This help channel has been closed. Feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.