背景
我需要在multiprocessing.ThreadPool中运行一个multiprocessing.Process。这似乎很奇怪，但这是我找到的唯一处理使用c++共享库可能导致的段错误的方法。如果发生段错误，则会杀死进程，我可以检查进程的exitcode并进行处理。
问题
一段时间后，在我尝试加入进程时会出现死锁。
以下是我的代码的简单版本：
import sys, time, multiprocessing
from multiprocessing.pool import ThreadPool
def main():
# Launch 8 workers
pool = ThreadPool(8)
it = pool.imap(run, range(500))
while True:
try:
it.next()
except StopIteration:
break
def run(value):
# Each worker launch it own Process
process = multiprocessing.Process(target=run_and_might_segfault, args=(value,))
process.start()
while process.is_alive():
sys.stdout.write('.')
sys.stdout.flush()
time.sleep(0.1)
# Will never join after a while, because of a mystery deadlock
process.join()
# Deals with process.exitcode to log errors
def run_and_might_segfault(value):
# Load a shared library and do stuff (could throw c++ exception, segfault ...)
print(value)
if __name__ == '__main__':
main()
以下是可能的输出：
➜ ~ python m.py
..0
1
........8
.9
.......10
......11
........12
13
........14
........16
........................................................................................
正如您所看到的，在几次迭代后，process.is_alive()始终为真，进程永远不会加入。
如果我按下CTRL-C键，则会得到以下堆栈跟踪：
Traceback (most recent call last):
File "/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/pool.py", line 680, in next
item = self._items.popleft()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "m.py", line 30, in <module>
main()
File "m.py", line 9, in main
it.next()
File "/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5 /lib/python3.5/multiprocessing/pool.py", line 684, in next
self._cond.wait(timeout)
File "/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5 /lib/python3.5/threading.py", line 293, in wait
waiter.acquire()
KeyboardInterrupt
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5 /lib/python3.5/multiprocessing/popen_fork.py", line 29, in poll
pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt
PS 我在macOS上使用Python 3.5.2。
非常感谢任何形式的帮助。
Edit
我尝试使用Python 2.7，它可以正常工作。这可能只是Python 3.5的问题？