大家晚上好,
这个问题让我有点尴尬,因为我知道我应该自己能够得到答案。然而,我的Python知识仅仅比零略微多一点,所以我需要从比我更有经验的人那里寻求帮助...
以下代码来自于最近编辑的一本书中Norvig's "Natural Language Corpus Data"一章,它的目的是将句子“likethisone”转换为“[like, this, one]”(也就是正确分割单词)...
我已经将所有代码移植到C#中(实际上是自己重写了程序),但是函数segment
让我非常困惑,甚至无法理解它的语法。请问是否有人能够帮我将其翻译成更易读的C#形式?
非常感谢您提前的帮助。
################ Word Segmentation (p. 223)
@memo
def segment(text):
"Return a list of words that is the best segmentation of text."
if not text: return []
candidates = ([first]+segment(rem) for first,rem in splits(text))
return max(candidates, key=Pwords)
def splits(text, L=20):
"Return a list of all possible (first, rem) pairs, len(first)<=L."
return [(text[:i+1], text[i+1:])
for i in range(min(len(text), L))]
def Pwords(words):
"The Naive Bayes probability of a sequence of words."
return product(Pw(w) for w in words)
#### Support functions (p. 224)
def product(nums):
"Return the product of a sequence of numbers."
return reduce(operator.mul, nums, 1)
class Pdist(dict):
"A probability distribution estimated from counts in datafile."
def __init__(self, data=[], N=None, missingfn=None):
for key,count in data:
self[key] = self.get(key, 0) + int(count)
self.N = float(N or sum(self.itervalues()))
self.missingfn = missingfn or (lambda k, N: 1./N)
def __call__(self, key):
if key in self: return self[key]/self.N
else: return self.missingfn(key, self.N)
def datafile(name, sep='\t'):
"Read key,value pairs from file."
for line in file(name):
yield line.split(sep)
def avoid_long_words(key, N):
"Estimate the probability of an unknown word."
return 10./(N * 10**len(key))
N = 1024908267229 ## Number of tokens
Pw = Pdist(datafile('count_1w.txt'), N, avoid_long_words)
return max(candidates, key=Pwords)
吗?这段代码将每个对象使用Pwords()
函数转换为数字,然后找到其中的最大值。现在尝试将生成器表达式中的candidates = ([first]+segment(rem) for first,rem in splits(text))
展开成一个 for 循环。实际上,你可以使用yield return
技巧。 - Hamish Grubijan