我有一个模块和一个模块加载器。该模块使用了numba。
当我使用importlib
进行动态加载后调用该模块的函数时,会出现多个错误。
当我使用static
导入并调用该函数时,一切正常。
当我不使用numba时,静态和动态导入都能正常工作。
以下是一个简单的示例来重现这个问题:
模块 mymod.py
:
from numba import jit
@jit(nopython=True)
def process():
a = 5
res = a + 5
return res
可靠的模块加载器:runb.py(静态导入)
import mymod
def main():
res = mymod.process()
print(res)
if __name__=="__main__":
main()
模块加载器出现问题:
run.py
(动态导入)from pathlib import Path
import importlib.util
def main():
module_path = "./mymod.py"
spec = importlib.util.spec_from_file_location(Path(module_path).name, module_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
res = module.process()
print(res)
if __name__=="__main__":
main()
软件环境:
- Ubuntu 19.04
- Python 3.7.5
- pip 18.1
- Numba 0.51.2
- llvmlite 0.34.0
文件夹结构:
- dumb/venv
- dumb/mymod.py
- dumb/run.py
- dumb/runb.py
我正在解决这个问题,但在互联网上没有找到任何有用的信息。
以下是错误信息:
Traceback (most recent call last):
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/environment.py", line 21, in from_fndesc
return cls._memo[fndesc.env_name]
File "/usr/lib/python3.7/weakref.py", line 137, in __getitem__
o = self.data[key]()
KeyError: '_ZN08NumbaEnv5mymod2py11process$241E'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/funcdesc.py", line 93, in lookup_module
return sys.modules[self.modname]
KeyError: 'mymod.py'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "run.py", line 16, in <module>
main()
File "run.py", line 10, in main
res = module.process()
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/dispatcher.py", line 434, in _compile_for_args
raise e
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/dispatcher.py", line 367, in _compile_for_args
return self.compile(tuple(argtypes))
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler_lock.py", line 32, in _acquire_compile_lock
return func(*args, **kwargs)
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/dispatcher.py", line 819, in compile
cres = self._compiler.compile(args, return_type)
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/dispatcher.py", line 78, in compile
status, retval = self._compile_cached(args, return_type)
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/dispatcher.py", line 92, in _compile_cached
retval = self._compile_core(args, return_type)
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/dispatcher.py", line 110, in _compile_core
pipeline_class=self.pipeline_class)
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler.py", line 627, in compile_extra
return pipeline.compile_extra(func)
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler.py", line 363, in compile_extra
return self._compile_bytecode()
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler.py", line 425, in _compile_bytecode
return self._compile_core()
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler.py", line 405, in _compile_core
raise e
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler.py", line 396, in _compile_core
pm.run(self.state)
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler_machinery.py", line 341, in run
raise patched_exception
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler_machinery.py", line 332, in run
self._runPass(idx, pass_inst, state)
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler_lock.py", line 32, in _acquire_compile_lock
return func(*args, **kwargs)
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler_machinery.py", line 291, in _runPass
mutated |= check(pss.run_pass, internal_state)
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler_machinery.py", line 264, in check
mangled = func(compiler_state)
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/typed_passes.py", line 442, in run_pass
NativeLowering().run_pass(state)
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/typed_passes.py", line 369, in run_pass
metadata=metadata)
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/lowering.py", line 37, in __init__
self.env = Environment.from_fndesc(self.fndesc)
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/environment.py", line 23, in from_fndesc
inst = cls(fndesc.lookup_globals())
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/funcdesc.py", line 81, in lookup_globals
return self.global_dict or self.lookup_module().__dict__
File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/funcdesc.py", line 96, in lookup_module
f"can't compile {self.qualname}: "
ModuleNotFoundError: can't compile process: import of module mymod.py failed
spec_from_file_location(name, path)
的名称应该是mymod
,而不是mymod.py
。 - Niklas Mertschspec = importlib.util.spec_from_file_location("mymod", module_path)
但问题仍然存在 :/ - SamoloHKeyError: 'mymod.py'
错误吗? - Niklas Mertsch