functools.partial与multiprocessing.Pool.map不兼容吗？

Question

我有一段代码，简化后看起来像这样：

run = functools.partial(run, grep=options.grep, print_only=options.print_only, force=options.force)

if not options.single and not options.print_only and options.n > 0:
    pool = multiprocessing.Pool(options.n)
    Map = pool.map
else: Map = map

for f in args:
    with open(f) as fh: Map(run, fh)

try:
    pool.close()
    pool.join()
except NameError: pass

如果我在单进程模式下运行它，那么一切都很好，但是在出现如下错误时会失败

TypeError: type 'partial' takes at least one argument

通过多进程模块，长调用栈混杂在一起。发生了什么？

我正在使用Python 2.6.1。

- pythonic metaphor

Answer 1

谷歌告诉我这是Python中的错误，在Py3k中已经修复。据说是由于partial无法被pickle化导致的。

有一个解决方法。

- Katriel

---

网页存档链接现在也失效了。 - RunOrVeith

Answer 2

它的工作原理是：

from multiprocessing.dummy import Pool as ThreadPool

def executeFunction(server,function):
    print "Server: %s and function: %s" % (server,function)


def executeParallel(servers,function, threads=10):
    pool = ThreadPool(threads)
    functionArguments=partial(executeFunction, function=function)
    results = pool.map(functionArguments, servers)
    pool.close()
    pool.join()
    return results

executeParallel(serverList, "Install")

- Diego Pereira