我希望从我提到的示例句子中获取二元组和三元组。
我的代码对于二元组很好用。然而,它无法捕获数据中的三元组(例如,人类计算机交互,在我的句子中被提及了5次)。
下面是使用Gensim中的Phrases的代码,方法1如下所述。
请帮我解决获取三元组的问题。
我正在遵循Gensim的示例文档。
我的代码对于二元组很好用。然而,它无法捕获数据中的三元组(例如,人类计算机交互,在我的句子中被提及了5次)。
下面是使用Gensim中的Phrases的代码,方法1如下所述。
from gensim.models import Phrases
documents = ["the mayor of new york was there", "human computer interaction and machine learning has now become a trending research area","human computer interaction is interesting","human computer interaction is a pretty interesting subject", "human computer interaction is a great and new subject", "machine learning can be useful sometimes","new york mayor was present", "I love machine learning because it is a new subject area", "human computer interaction helps people to get user friendly applications"]
sentence_stream = [doc.split(" ") for doc in documents]
bigram = Phrases(sentence_stream, min_count=1, threshold=1, delimiter=b' ')
trigram = Phrases(bigram_phraser[sentence_stream])
for sent in sentence_stream:
bigrams_ = bigram_phraser[sent]
trigrams_ = trigram[bigrams_]
print(bigrams_)
print(trigrams_)
方法2 我甚至尝试使用Phraser和Phrases,但它没有起作用。
from gensim.models import Phrases
from gensim.models.phrases import Phraser
documents = ["the mayor of new york was there", "human computer interaction and machine learning has now become a trending research area","human computer interaction is interesting","human computer interaction is a pretty interesting subject", "human computer interaction is a great and new subject", "machine learning can be useful sometimes","new york mayor was present", "I love machine learning because it is a new subject area", "human computer interaction helps people to get user friendly applications"]
sentence_stream = [doc.split(" ") for doc in documents]
bigram = Phrases(sentence_stream, min_count=1, threshold=2, delimiter=b' ')
bigram_phraser = Phraser(bigram)
trigram = Phrases(bigram_phraser[sentence_stream])
for sent in sentence_stream:
bigrams_ = bigram_phraser[sent]
trigrams_ = trigram[bigrams_]
print(bigrams_)
print(trigrams_)
请帮我解决获取三元组的问题。
我正在遵循Gensim的示例文档。
threshold
值会发生什么吗?因为对我来说不是很清楚。 - user8566323delimiter=b' '
必须是二进制格式。感谢提醒。 - Max