我希望实现的目标:
并行化一个函数,每次调用生成一定数量的线程,像这样:
- PROCESS01 -> 16 Threads
- PROCESS02 -> 16 Threads
- ...
- PROCESSn -> 16 Threads
代码:
with multiprocessing.Pool(4) as process_pool:
results = process_pool.map(do_stuff, [drain_queue()])
当 drain_queue()
返回一个项目列表时,
do_stuff(item_list):
print('> PID: ' + str(os.getpid()))
with concurrent.futures.ThreadPoolExecutor(max_workers=16) as executor:
result_dict = {executor.submit(thread_function, item): item for item in item_list}
for future in concurrent.futures.as_completed(result_dict):
pass
而thread_function()
会处理传递给它的每个项目。
然而,当执行代码时,输出如下:
> PID: 1000
(WAITS UNTIL THE PROCESS FINISHES, THEN START NEXT)
> PID: 2000
(WAITS UNTIL THE PROCESS FINISHES, THEN START NEXT)
> PID: 3000
(WAITS UNTIL THE PROCESS FINISHES, THEN START NEXT)
> PID: 3000
(WAITS UNTIL THE PROCESS FINISHES, THEN START NEXT)
我错过了什么吗?我无法弄清楚为什么它不能按预期工作。 谢谢!