问题描述
我正在尝试从磁盘加载一个自定义预训练模型和自定义管道作为 spacy 3.0 中的管道:
工厂的代码是这样的:
@CustomEng.factory("ner-crf")
def create_my_component(nlp,name):
crf_extractor = CRFExtractor().from_disk("path-to-model")
return CRFEntityExtractor(nlp,crf_extractor=crf_extractor)
然后我像这样将“ner-crf”添加到我的语言课程中:
nlp = spacy.blank('custom-eng')
nlp.add_pipe('ner-crf')
nlp.to_disk('../model')
我认为有一件事可能是相关的:当我使用 to_disk
来保存 nlp
对象时,保存的文件中没有 ner-crf
包(我刚刚添加的管道)对象。
然后我运行这个 CLI 命令来评估 NER 管道:
python -m spacy evaluate ../model/ ../corpus/dev.spacy --output ../model/metrics.json --gpu-id 0 --code ../../../spacy_utils/custom-eng/__init__.py
但我收到此错误:
Traceback (most recent call last):
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/runpy.py",line 194,in _run_module_as_main
return _run_code(code,main_globals,None,File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/runpy.py",line 87,in _run_code
exec(code,run_globals)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/spacy/__main__.py",line 4,in <module>
setup_cli()
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/spacy/cli/_util.py",line 69,in setup_cli
command(prog_name=COMMAND)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/click/core.py",line 829,in __call__
return self.main(*args,**kwargs)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/click/core.py",line 782,in main
rv = self.invoke(ctx)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/click/core.py",line 1259,in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/click/core.py",line 1066,in invoke
return ctx.invoke(self.callback,**ctx.params)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/click/core.py",line 610,in invoke
return callback(*args,**kwargs)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/typer/main.py",line 497,in wrapper
return callback(**use_params) # type: ignore
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/spacy/cli/evaluate.py",line 42,in evaluate_cli
evaluate(
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/spacy/cli/evaluate.py",line 75,in evaluate
nlp = util.load_model(model)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/spacy/util.py",line 326,in load_model
return load_model_from_path(Path(name),**kwargs)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/spacy/util.py",line 392,in load_model_from_path
return nlp.from_disk(model_path,exclude=exclude)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/spacy/language.py",line 1883,in from_disk
util.from_disk(path,deserializers,exclude)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/spacy/util.py",line 1176,in from_disk
reader(path / key)
File "/home/marzi/anaconda3/envs/spacy-tutorial/lib/python3.8/site-packages/spacy/language.py",line 1877,in <lambda>
deserializers[name] = lambda p,proc=proc: proc.from_disk(
TypeError: from_disk() got an unexpected keyword argument 'exclude'
我使用的自定义 NER 类属于 spacy-crfsuite 库,它在 spacy 2 中运行良好,但它们还没有用于 Spacy 3 的示例代码,所以我试图让它自己在 spacy 3.0 中运行。
>解决方法
从 spaCy v3.0 开始,管道组件应在其 exclude
方法中支持 to_disk
关键字。您可以将 exclude
关键字添加到您的函数中,给它一个默认值,并且根本不在函数体中使用它的值,这个错误应该得到解决。
为了完整起见,这里是从 v2 到 v3 的迁移指南,其中可能包含一些其他有趣的提示:https://spacy.io/usage/v3#migrating