如何将CoNLL格式转换为spacy格式。

Question

我目前正在开发一个命名实体识别模型。我有一堆以CoNLL格式存储的数据需要转换为Spacy格式。在CoNLL中，每个句子的每个单词旁边都有一个标签。在Spacy中，标签只会显示给那些实际上有标签的单词。 如何从以下格式（CoNLL）进行转换？

From    O
2001    B-DateTime
to  I-DateTime
2004    I-DateTime
,   O
I   O
was O
a   O
stagehand   O
for O
Hartford    B-Company
Stage   I-Company
Company O
.   O

转换为以下格式（Spacy）

TRAIN_DATA = [('what is the price of polo?', {'entities': [(21, 25, 'PrdName')]}), 
              ('what is the price of ball?', {'entities': [(21, 25, 'PrdName')]}), 
              ('what is the price of jegging?', {'entities': [(21, 28, 'PrdName')]}), 
              ('what is the price of t-shirt?', {'entities': [(21, 28, 'PrdName')]}), 
              ('what is the price of jeans?', {'entities': [(21, 26, 'PrdName')]}), 
              ('what is the price of bat?', {'entities': [(21, 24, 'PrdName')]}), 
              ('what is the price of shirt?', {'entities': [(21, 26, 'PrdName')]}), 
              ('what is the price of bag?', {'entities': [(21, 24, 'PrdName')]}), 
              ('what is the price of cup?', {'entities': [(21, 24, 'PrdName')]}), 
              ('what is the price of jug?', {'entities': [(21, 24, 'PrdName')]}), 
              ('what is the price of plate?', {'entities': [(21, 26, 'PrdName')]}), 
              ('what is the price of glass?', {'entities': [(21, 26, 'PrdName')]}),
              ('what is the price of watch?', {'entities': [(21, 26, 'PrdName')]})]

- Big Code Guy

Answer 1

只需使用spacy convert。

spacy convert input.conll -c conll ./output/

请注意，默认情况下会生成一个二进制的.spacy文件。JSON格式在v3中已被弃用，对于大多数情况并不是很有用。

- polm23