🔒 ProcessPoolExecutor worker giving messy traceback

[analog island]

Hello,

I have a minor issue with ProcessPoolExecutors that I wasn't able to find a solution to.

I am running an aiohttp server that needs to do some short-term cpu-bound function calls (bcrypt).

So I decided to use one long-running ProcessPoolExecutor object for this to keep the main event loop free to respond to requests (since I need to account for a large volume of bcrypt calls).

My main server looks something like:

class Server:
def __init__(self):
  self.process_pool = ProcessPoolExecutor(...)

def run(self):
  try:
    # exiting this context manager will call .shutdown()
    with self.process_pool:
      ...
  except (KeyboardInterrupt, SystemExit):
      ...

I am submitting tasks from request handlers like:

loop = asyncio.get_running_loop()
loop.run_in_executor(process_pool, ...)

the problem is when this gets interrupted with KeyboardInterrupt, I catch it and everything closes cleanly but these child processes all give out a messy traceback, which I will post in a below message.

The traceback repeats once per child process.

Please can someone explain to me why this is happening and help me fix it? Any help is much appreciated.

[royal oar]

@analog island

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[analog island]

The traceback whick repeats once per child process

Process SpawnProcess-1:
Traceback (most recent call last):
  File "C:\Users\...\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\process.py", line 314, in _bootstrap
    self.run()
  File "C:\Users\...\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\...\AppData\Local\Programs\Python\Python312\Lib\concurrent\futures\process.py", line 251, in _process_worker
    call_item = call_queue.get(block=True)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\...\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\queues.py", line 103, in get
    res = self._recv_bytes()
          ^^^^^^^^^^^^^^^^^^
  File "C:\Users\...\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\connection.py", line 216, in recv_bytes
    buf = self._recv_bytes(maxlength)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\...\AppData\Local\Programs\Python\Python312\Lib\multiprocessing\connection.py", line 321, in _recv_bytes
    waitres = _winapi.WaitForMultipleObjects(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt

[winter geode]

im actually not sure how to deal with this, i found this SO answer https://stackoverflow.com/a/31521453/4849865 and it looks like people have been having this issue for forever

You are not getting a proper stack trace, because the exception is actually pickled from the subprocess (hence the name of the module: multiprocessing) your code is executing in and is then raised again in the manager process. Hence the stack trace shows up as if it was local (while it is actually an exception raised in the subprocess).

[analog island]

Thanks for helping, I feel in over my head with this

I since found down a rabbit hole from the link you sent, I can do

def init_worker():
    signal.signal(signal.SIGINT, signal.SIG_IGN)

and pass this function as initialiser kwarg when creating the ProcessPoolExecutor which appears to solve my issue, my understanding is it tells workers to ignore KeyboardInterrupt
(but I don't really fully understand what is going on under the hood in this case)

[unreal barn]

Might be better to not use SIG_ING, but to supply a little function which prints "interrupted!" or something and calls sys.exit(1).

A signal handler function looks like:

def onsig(signal, frame):
  print(f'interrupted by {signal=}')
  sys.exit(1)

and just supply onsig instead of signal.SIG_IGN.

[analog island]

I see, that's definitely an improvement

I need to get a better understanding of this signal stuff. I kinda understand some ideas but I'm not completely familiar. Thanks for the help 😄

[unreal barn]

It's a crude interprocess mechanism in UNIX. And apparently simulated to some degree in Windows.

You can send a "signal" to an arbitrary process, it's just a number. It interrupts the process and runs your signal handler (or is ignored with SIG_IGN, or has the default behaviour SIG_DFLT).

KeyboardInterrupt is the signal sent for ctrl-c (or whatever your "interrupt keystroke is).

[royal oar]

Python help channel closed for inactivity

This help channel has been closed. Feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.