我正在使用concurrent.futures实现多进程。 我收到了一个queue.Full错误,这很奇怪,因为我只分配了10个作业。
A_list = [np.random.rand(2000, 2000) for i in range(10)]
with ProcessPoolExecutor() as pool:
pool.map(np.linalg.svd, A_list)
错误:
Exception in thread Thread-9:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/threading.py", line 921, in _bootstrap_inner
self.run()
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/threading.py", line 869, in run
self._target(*self._args, **self._kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/concurrent/futures/process.py", line 251, in _queue_management_worker
shutdown_worker()
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/concurrent/futures/process.py", line 209, in shutdown_worker
call_queue.put_nowait(None)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/queues.py", line 131, in put_nowait
return self.put(obj, False)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/multiprocessing/queues.py", line 82, in put
raise Full
queue.Full
shutdown_worker
的失败只会运行一次-因此,您需要追踪的真正问题是为什么会发生这种情况。 - dano