Python的多进程无法按预期工作

Question

我在网上找到了这段代码，预期结果应该显示当前正在处理的进程名称，但是当我运行代码时，它只给出了结果而没有名称。我运行的是Windows 7 Python 2.7.3。

代码：

import multiprocessing

def do_calculation(data):
    return data * 2

def start_process():
    print 'Starting', multiprocessing.current_process().name

if __name__ == '__main__':
    inputs = list(range(10))
    print 'Input   :', inputs

    builtin_outputs = map(do_calculation, inputs)
    print 'Built-in:', builtin_outputs

    pool_size = multiprocessing.cpu_count() * 2
    pool = multiprocessing.Pool(processes=pool_size,
                                initializer=start_process,
                                )
    pool_outputs = pool.map(do_calculation, inputs)
    pool.close() # no more tasks
    pool.join()  # wrap up current tasks

    print 'Pool    :', pool_outputs

预期结果：

$ python multiprocessing_pool.py

Input   : [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
Built-in: [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
Starting PoolWorker-11
Starting PoolWorker-12
Starting PoolWorker-13
Starting PoolWorker-14
Starting PoolWorker-15
Starting PoolWorker-16
Starting PoolWorker-1
Starting PoolWorker-2
Starting PoolWorker-3
Starting PoolWorker-4
Starting PoolWorker-5
Starting PoolWorker-8
Starting PoolWorker-9
Starting PoolWorker-6
Starting PoolWorker-10
Starting PoolWorker-7
Pool    : [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]

- user2569466

---

你为什么期望那种情况会发生？根据你展示的代码，你所贴出的输出看起来是符合预期的行为。 - Gian

我唯一的猜测是，您实际上希望您的 do_calculation 函数体看起来像这样：return (data * 2, multiprocessing.current_process().name)，其中它以元组形式返回 data * 2 和工作进程名称。 - sberry

你尝试过检查multiprocessing.cpu_count()返回什么吗？ - fsw

Answer 1

如果我在控制台中运行以下代码，而不是在使用PyCharm后又回到的IDLE中：

from multiprocessing import *

def main():
    data = list(range(10))
    print('Data:', data)

    result = list(map(calculate, data))
    print('Map: ', result)

    pool = Pool(cpu_count() * 2, initialize)
    result = pool.map(calculate, data, 1)
    pool.close()
    pool.join()

    print('Pool:', result)

def calculate(number):
    return number * 2

def initialize():
    print('Starting', current_process().name)

if __name__ == '__main__':
    main()

以下内容将按照您的预期打印到控制台屏幕上（PoolWorkers 的顺序随机）：

C:\Users\schappell\Desktop>stack_overflow.py
Data: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
Map:  [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
Starting PoolWorker-1
Starting PoolWorker-13
Starting PoolWorker-2
Starting PoolWorker-3
Starting PoolWorker-8
Starting PoolWorker-5
Starting PoolWorker-14
Starting PoolWorker-16
Starting PoolWorker-15
Starting PoolWorker-12
Starting PoolWorker-6
Starting PoolWorker-9
Starting PoolWorker-4
Starting PoolWorker-10
Starting PoolWorker-7
Starting PoolWorker-11
Pool: [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]

C:\Users\schappell\Desktop>

---

作为附注，尝试运行以下代码（启用和禁用 time.sleep）：

from multiprocessing import *
import time

def main():
    data = list(range(10))
    print('Data:', data)
    print('Map: ', list(map(calculate, data)))
    pool = Pool(cpu_count() * 2, initialize)
##    time.sleep(1)
    print('Pool:', pool.map(calculate, data, 1))
    pool.close()
    pool.join()

def calculate(number):
    return number * 2

def initialize():
    print('Starting', current_process().name)

if __name__ == '__main__':
    main()

没有休眠，你可能会得到类似这样的结果（显示您的池尚未完全创建）：

C:\Users\schappell\Desktop>stack_overflow.py
Data: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
Map:  [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
Starting PoolWorker-2
Pool: [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
Starting PoolWorker-1
Starting PoolWorker-10
Starting PoolWorker-9
Starting PoolWorker-6
Starting PoolWorker-14
Starting PoolWorker-13
Starting PoolWorker-5
Starting PoolWorker-4
Starting PoolWorker-3
Starting PoolWorker-8
Starting PoolWorker-11
Starting PoolWorker-12
Starting PoolWorker-16
Starting PoolWorker-7
Starting PoolWorker-15

C:\Users\schappell\Desktop>

- Noctis Skytower