Python多进程池在Linux和Windows之间有不同的行为。
当通过工作人数运行map方法时,在Linux中,它会在您作为参数提供的特定函数的范围内运行该进程, 但是在Windows中,每个工作者都在父进程的范围内运行,并再次使用不应该的代码。
例如:(flask仅用于使其类似于我的代码)
from multiprocessing import Pool, Event
from flask import Flask
print(">>> This code running for every each worker")
app = Flask(__name__)
terminating = None
def f(**kwargs):
print("f()")
x = kwargs.pop("x", 1)
print(x * x)
return x * x
def worker_warpper(arg):
func, kwargs = arg
return func(**kwargs)
def initializer(terminating_):
global terminating
terminating = terminating_
@app.route('/check', methods=['GET'])
def check():
with Pool(processes=3) as pool:
ls = [(f, {"x": 2}), (f, {"x": 5}), (f, {"x": 6})]
pool_map = pool.map(worker_warpper, ls)
return "Finished"
if __name__ == "__main__":
print("Listening...")
app.run(port=5151, host='0.0.0.0')
这段代码应该在三个不同的进程中并行运行函数"f",但它重新运行了顶部的打印内容。(不完全是每个进程都再次运行 - 但是运行"f"的次数与重复运行顶部的打印次数之间存在关系)。只有在Windows中,Linux中只有"f"会再次运行。
输出:(Linux)
>>> This code running for new worker (not all of the workers)
Listening
...
* Running on http://0.0.0.0:5151/ (Press CTRL+C to quit)
f()
4
f()
25
f()
36
127.0.0.1 - -
[29/Jan/2017 11:46:26] "GET /check HTTP/1.1" 200 -
输出:(Windows)
>>> This code running for new worker (not all of the workers)
Listening
...
* Running on http://0.0.0.0:5151/ (Press CTRL+C to quit)
>>> This code running for new worker (not all of the workers)
f()
4
f()
25
f()
36
127.0.0.1 - -
[29/Jan/2017 11:46:26] "GET /check HTTP/1.1" 200 -
为什么Linux和Windows的行为不同?我该怎么办?
如果不清楚,请告诉我,我会用不同的方式尝试解释。
谢谢!