答案的核心是“使用管道组件“beam_ner”,并查看EntityRecognizer.pyx代码。然后有单元测试test_ner.py test_beam_ner_scores(),几乎展示了如何做到这一点。如果您想查看如何修改配置文件cfg,请保存模型(如下所示make_nlp()),并查看保存的模型config.cfg。
问题在于它仅适用于单元测试生成的“model”。对于我的真实模型(5000个文档,每个文本约4k,训练NER F得分约为75%),它表现不佳。当我说“表现不佳”时,我是指“贪婪搜索”可以找到我的实体,但“波束搜索”会报告数百个标记(甚至包括标点符号),其“分数”通常为0.013。根据偏移量,这些标记通常来自文档的一个小部分。
这很令人沮丧,因为我相信spacy train(针对'beam_ner')使用相同的代码来“验证”训练迭代,并且训练报告的得分几乎还不错(好吧,比Spacy 2低10%,但这也发生在使用'ner'和'beam_ner'进行训练时)。
因此,我发布这篇文章,希望有人有更好的运气或指出我做错了什么。
到目前为止,Spacy3 对我来说是一个重大灾难:无法获得置信度,也不能再使用 GPU(我只有 6GB),基于 Ray 的并行化不起作用(在 Windows 上 = 实验性质),而使用“transformer”模型进行实体命名识别的训练分数比 Spacy 2 差了 10%。
代码
import spacy
from spacy.lang.en import English
from spacy.language import Language
from spacy.tokens import Doc
from spacy.training import Example
TRAIN_DATA = [
("Who is Shaka Khan?", {"entities": [(7, 17, "PERSON")]}),
("I like London and Berlin.", {"entities": [(7, 13, "LOC"), (18, 24, "LOC")]}),
("You like Paris and Prague.", {"entities": [(9, 14, "LOC"), (19, 25, "LOC")]}),
]
def make_nlp(model_dir):
nlp = English()
config = { "beam_width": 32, "beam_density": 0.001 }
ner = nlp.add_pipe("beam_ner", config=config)
train_examples = []
for text, annotations in TRAIN_DATA:
train_examples.append(Example.from_dict(nlp.make_doc(text), annotations))
for ent in annotations.get("entities"):
ner.add_label(ent[2])
optimizer = nlp.initialize()
losses = {}
nlp.update(train_examples, sgd=optimizer, losses=losses)
nlp.to_disk(model_dir)
print("Saved model to", model_dir)
return nlp
def test_greedy(nlp, text):
doc = nlp(text)
print("GREEDY search");
for ent in doc.ents:
print("Greedy offset=", ent.start_char, "-", ent.end_char, ent.label_, "text=", ent.text)
def test_beam(nlp, text):
ner = nlp.get_pipe("beam_ner")
doc = nlp.make_doc(text)
docs = [doc]
beams = ner.predict(docs)
print("BEAM search, labels", ner.labels);
scores = ner.scored_ents(beams)[0]
for ent, sco in scores.items():
tok = doc[ent[0]]
lbl = ent[2]
spn = doc[ent[0]: ent[1]]
print('Beam-search', ent[0], ent[1], 'offset=', tok.idx, lbl, 'score=', sco,
'text=', spn.text.replace('\n', ' '))
MODEL_DIR = "./test_model"
TEST_TEXT = "I like London and Paris."
if __name__ == "__main__":
nlp = spacy.load(MODEL_DIR)
test_greedy(nlp, TEST_TEXT)
test_beam (nlp, TEST_TEXT)
结果应该看起来像这样(在重复使用make_nlp生成可用的“模型”之后):
GREEDY search
Greedy offset= 7 - 13 LOC text= London
Greedy offset= 18 - 23 LOC text= Paris
BEAM search, labels ('LOC', 'PERSON')
Beam-search 2 3 offset= 7 LOC score= 0.5315668466265199 text= London
Beam-search 4 5 offset= 18 LOC score= 0.7206478212662492 text= Paris
Beam-search 0 1 offset= 0 LOC score= 0.4679245513356703 text= I
Beam-search 3 4 offset= 14 LOC score= 0.4670399792743775 text= and
Beam-search 5 6 offset= 23 LOC score= 0.2799470367073933 text= .
Beam-search 1 2 offset= 2 LOC score= 0.21658368070744227 text= like