动态python模块导入和numba

编程问答 2022-06-05

问题描述

我有一个模块和一个模块加载器。该模块使用numba。

使用importlib动态加载模块后调用模块的函数时，出现了一些错误。 当我执行静态导入并调用该函数时，一切正常。

当我不使用numba时，静态和动态导入都可以正常工作。

以下是重现该问题的简单示例：

模块：mymod.py 

from numba import jit

@jit(nopython=True)
def process():
    a = 5
    res = a + 5
    return res

起作用的模块加载器：runb.py（静态导入）

import mymod

def main():
    res = mymod.process()
    print(res)

if __name__=="__main__":
    main()

不起作用的模块加载器：run.py（动态导入）

from pathlib import Path
import importlib.util

def main():
    module_path = "./mymod.py"
    spec = importlib.util.spec_from_file_location(Path(module_path).name,module_path) 
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)

    res = module.process()

    print(res)

if __name__=="__main__":
    main()

软件环境：

  • Ubuntu 19.04
  • Python 3.7.5
  • 点 18.1
  • Numba 0.51.2
  • llvmlite 0.34.0

文件夹视图：

  • dumb / venv
  • dumb / mymod.py
  • dumb / run.py
  • dumb / runb.py

我正在为这个问题而苦苦挣扎，没有在互联网上找到任何东西。

错误以下：

Traceback (most recent call last):
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/environment.py",line 21,in from_fndesc
    return cls._memo[fndesc.env_name]
  File "/usr/lib/python3.7/weakref.py",line 137,in __getitem__
    o = self.data[key]()
KeyError: '_ZN08NumbaEnv5mymod2py11process$241E'

During handling of the above exception,another exception occurred:

Traceback (most recent call last):
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/funcdesc.py",line 93,in lookup_module
    return sys.modules[self.modname]
KeyError: 'mymod.py'

During handling of the above exception,another exception occurred:

Traceback (most recent call last):
  File "run.py",line 16,in <module>
    main()
  File "run.py",line 10,in main
    res = module.process()
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/dispatcher.py",line 434,in _compile_for_args
    raise e
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/dispatcher.py",line 367,in _compile_for_args
    return self.compile(tuple(argtypes))
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler_lock.py",line 32,in _acquire_compile_lock
    return func(*args,**kwargs)
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/dispatcher.py",line 819,in compile
    cres = self._compiler.compile(args,return_type)
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/dispatcher.py",line 78,in compile
    status,retval = self._compile_cached(args,line 92,in _compile_cached
    retval = self._compile_core(args,line 110,in _compile_core
    pipeline_class=self.pipeline_class)
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler.py",line 627,in compile_extra
    return pipeline.compile_extra(func)
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler.py",line 363,in compile_extra
    return self._compile_bytecode()
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler.py",line 425,in _compile_bytecode
    return self._compile_core()
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler.py",line 405,in _compile_core
    raise e
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler.py",line 396,in _compile_core
    pm.run(self.state)
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler_machinery.py",line 341,in run
    raise patched_exception
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler_machinery.py",line 332,in run
    self._runPass(idx,pass_inst,state)
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler_lock.py",**kwargs)
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler_machinery.py",line 291,in _runPass
    mutated |= check(pss.run_pass,internal_state)
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/compiler_machinery.py",line 264,in check
    mangled = func(compiler_state)
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/typed_passes.py",line 442,in run_pass
    NativeLowering().run_pass(state)
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/typed_passes.py",line 369,in run_pass
    Metadata=Metadata)
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/lowering.py",line 37,in __init__
    self.env = Environment.from_fndesc(self.fndesc)
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/environment.py",line 23,in from_fndesc
    inst = cls(fndesc.lookup_globals())
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/funcdesc.py",line 81,in lookup_globals
    return self.global_dict or self.lookup_module().__dict__
  File "mypath/dumb/venv/lib/python3.7/site-packages/numba/core/funcdesc.py",line 96,in lookup_module
    f"can't compile {self.qualname}: "
ModuleNotFoundError: can't compile process: import of module mymod.py Failed

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）