logo标签列表

在一个单词中检测音节

nlpspell-checkinghyphenation
156

我需要找到一种相当有效的方法来检测单词中的音节。例如，

Invisible -> in-vi-sib-le

有一些可以使用的划分音节规则：

V CV VC CVC CCV CCCV CVCC

*其中V表示元音，C表示辅音。例如，

Pronunciation (5 Pro-nun-ci-a-tion; CV-CVC-CV-V-CVC)

我已经尝试了一些方法，包括使用正则表达式（只有在您想要计算音节数目时才有用）或硬编码规则定义（一种效率非常低下的暴力方法），最后使用有限状态自动机（并没有得到任何有用的结果）。

我的应用程序的目的是创建给定语言中所有音节的字典。稍后将使用此字典进行拼写检查应用程序（使用贝叶斯分类器）和文本到语音合成。

除了以前的方法之外，如果有人能够给我提供解决此问题的替代方法，我将不胜感激。

我使用Java工作，但对于C / C ++，C＃，Python，Perl等的任何提示都适用于我。

你是想要实际的分割点还是只是单词中音节的数量？如果是后者，考虑在文本到语音字典中查找单词并计算编码元音音素的音位数。 - Adrian McCarthy
最有效的方式（从计算角度而言，不是存储角度），我猜想应该是使用Python字典，以单词作为键，音节数作为值。然而，你仍然需要一个备选方案来处理未在字典中出现的单词。如果你找到这样的字典，请告诉我！ - Brōtsyorfuzthrāx
17个回答
4

Perl有Lingua::Phonology::Syllable模块。你可以尝试使用它，或者研究其算法。我在那里看到了一些其他较旧的模块。

我不明白为什么正则表达式只能给出音节计数。你应该能够使用捕获括号获取音节本身。假设你能构造一个有效的正则表达式。

2
你可以尝试使用Spacy Syllables。它适用于Python 3.9：
设置：
pip install spacy
pip install spacy_syllables
python -m spacy download en_core_web_md

代码：

import spacy
from spacy_syllables import SpacySyllables
nlp = spacy.load('en_core_web_md')
syllables = SpacySyllables(nlp)
nlp.add_pipe('syllables', after='tagger')


def spacy_syllablize(word):
    token = nlp(word)[0]
    return token._.syllables


for test_word in ["trampoline", "margaret", "invisible", "thought", "Pronunciation", "couldn't"]:
    print(f"{test_word} -> {spacy_syllablize(test_word)}")

输出：

trampoline -> ['tram', 'po', 'line']
margaret -> ['mar', 'garet']
invisible -> ['in', 'vis', 'i', 'ble']
thought -> ['thought']
Pronunciation -> ['pro', 'nun', 'ci', 'a', 'tion']
couldn't -> ['could']
SpacySyllables还算不错，但需要注意的是它并不完美。例如"eighty"返回['eighty']，而"universal"则返回['uni', 'ver', 'sal']。这是因为底层库(Pyphen)默认将第一个和最后一个音节设置为2个字符。 - dacort
2

我无法找到一个足够好的方法来计算音节，所以我自己设计了一种方法。

您可以在这里查看我的方法: https://dev59.com/DWDVa4cB1Zd3GeqPZxYu#32784041

我使用字典和算法相结合的方法来计算音节。

您可以在这里查看我的库: https://github.com/troywatson/Lawrence-Style-Checker

我刚刚测试了我的算法，成功率达到了99.4%！

Lawrence lawrence = new Lawrence();

System.out.println(lawrence.getSyllable("hyphenation"));
System.out.println(lawrence.getSyllable("computer"));

输出：

4
3
1
通常，对于工具或库的链接，应该附带使用说明、关于链接资源如何适用于问题的具体解释或一些示例代码，如果可能的话，最好都包含。 - IKavanagh
请查看语法高亮。在SO编辑器中有一个帮助按钮（问号），可以让您进入链接的页面。 - IKavanagh
链接已失效，该库似乎不再可用。 - Aidan
2
经过大量测试和尝试使用连字包，我根据多个示例编写了自己的连字包。我还尝试了与连字词典交互的pyhyphen和pyphen包，但在许多情况下它们会产生错误的音节数。对于这种用例，nltk包太慢了。
我的Python实现是我编写的一个类的一部分，音节计数例程如下所示。它稍微高估了音节数，因为我仍然没有找到一个好的方法来解决无声词尾的问题。
该函数返回每个单词的音节比率，用于Flesch-Kincaid可读性评分。这个数字不必精确，只需足够接近一个估计值。
在我的第七代i7 CPU上，这个函数对于759个单词的样本文本需要1.1-1.2毫秒。
def _countSyllablesEN(self, theText):

    cleanText = ""
    for ch in theText:
        if ch in "abcdefghijklmnopqrstuvwxyz'’":
            cleanText += ch
        else:
            cleanText += " "

    asVow    = "aeiouy'’"
    dExep    = ("ei","ie","ua","ia","eo")
    theWords = cleanText.lower().split()
    allSylls = 0
    for inWord in theWords:
        nChar  = len(inWord)
        nSyll  = 0
        wasVow = False
        wasY   = False
        if nChar == 0:
            continue
        if inWord[0] in asVow:
            nSyll += 1
            wasVow = True
            wasY   = inWord[0] == "y"
        for c in range(1,nChar):
            isVow  = False
            if inWord[c] in asVow:
                nSyll += 1
                isVow = True
            if isVow and wasVow:
                nSyll -= 1
            if isVow and wasY:
                nSyll -= 1
            if inWord[c:c+2] in dExep:
                nSyll += 1
            wasVow = isVow
            wasY   = inWord[c] == "y"
        if inWord.endswith(("e")):
            nSyll -= 1
        if inWord.endswith(("le","ea","io")):
            nSyll += 1
        if nSyll < 1:
            nSyll = 1
        # print("%-15s: %d" % (inWord,nSyll))
        allSylls += nSyll

    return allSylls/len(theWords)
2
感谢 @joe-basirico 和 @tihamer，我已经将 @tihamer 的代码移植到了 Lua 5.1、5.2 和 luajit 2（很可能也能运行在其他版本的 Lua 上）:

countsyllables.lua

function CountSyllables(word)
  local vowels = { 'a','e','i','o','u','y' }
  local numVowels = 0
  local lastWasVowel = false

  for i = 1, #word do
    local wc = string.sub(word,i,i)
    local foundVowel = false;
    for _,v in pairs(vowels) do
      if (v == string.lower(wc) and lastWasVowel) then
        foundVowel = true
        lastWasVowel = true
      elseif (v == string.lower(wc) and not lastWasVowel) then
        numVowels = numVowels + 1
        foundVowel = true
        lastWasVowel = true
      end
    end

    if not foundVowel then
      lastWasVowel = false
    end
  end

  if string.len(word) > 2 and
    string.sub(word,string.len(word) - 1) == "es" then
    numVowels = numVowels - 1
  elseif string.len(word) > 1 and
    string.sub(word,string.len(word)) == "e" then
    numVowels = numVowels - 1
  end

  return numVowels
end

以下是一些有趣的测试，以确认其工作正常（尽可能地）：

countsyllables.tests.lua

require "countsyllables"

tests = {
  { word = "what", syll = 1 },
  { word = "super", syll = 2 },
  { word = "Maryland", syll = 3},
  { word = "American", syll = 4},
  { word = "disenfranchized", syll = 5},
  { word = "Sophia", syll = 2},
  { word = "End", syll = 1},
  { word = "I", syll = 1},
  { word = "release", syll = 2},
  { word = "same", syll = 1},
}

for _,test in pairs(tests) do
  local resultSyll = CountSyllables(test.word)
  assert(resultSyll == test.syll,
    "Word: "..test.word.."\n"..
    "Expected: "..test.syll.."\n"..
    "Result: "..resultSyll)
end

print("Tests passed.")
我添加了两个新的测试用例“End”和“I”。修复方法是不区分大小写地比较字符串。@joe-basirico和tihamer可能遇到同样的问题并希望更新他们的函数，因此在此提醒。 - josefnpat
@tihamer 美国有4个音节！ - josefnpat
0

我在这里提供一个在 R 中“可以”工作的解决方案。远非完美。

countSyllablesInWord = function(words)
  {
  #word = "super";
  n.words = length(words);
  result = list();
  for(j in 1:n.words)
    {
    word = words[j];
    vowels = c("a","e","i","o","u","y");
    
    word.vec = strsplit(word,"")[[1]];
    word.vec;
    
    n.char = length(word.vec);
    
    is.vowel = is.element(tolower(word.vec), vowels);
    n.vowels = sum(is.vowel);
    
    
    # nontrivial problem 
    if(n.vowels <= 1)
      {
      syllables = 1;
      str = word;
      } else {
              # syllables = 0;
              previous = "C";
              # on average ? 
              str = "";
              n.hyphen = 0;
        
              for(i in 1:n.char)
                {
                my.char = word.vec[i];
                my.vowel = is.vowel[i];
                if(my.vowel)
                  {
                  if(previous == "C")
                    {
                    if(i == 1)
                      {
                      str = paste0(my.char, "-");
                      n.hyphen = 1 + n.hyphen;
                      } else {
                              if(i < n.char)
                                {
                                if(n.vowels > (n.hyphen + 1))
                                  {
                                  str = paste0(str, my.char, "-");
                                  n.hyphen = 1 + n.hyphen;
                                  } else {
                                           str = paste0(str, my.char);
                                          }
                                } else {
                                        str = paste0(str, my.char);
                                        }
                              }
                     # syllables = 1 + syllables;
                     previous = "V";
                    } else {  # "VV"
                          # assume what  ?  vowel team?
                          str = paste0(str, my.char);
                          }
            
                } else {
                            str = paste0(str, my.char);
                            previous = "C";
                            }
                #
                }
        
              syllables = 1 + n.hyphen;
              }
  
      result[[j]] = list("syllables" = syllables, "vowels" = n.vowels, "word" = str);
      }
  
  if(n.words == 1) { result[[1]]; } else { result; }
  }

这是一些结果：
my.count = countSyllablesInWord(c("America", "beautiful", "spacious", "skies", "amber", "waves", "grain", "purple", "mountains", "majesty"));

my.count.df = data.frame(matrix(unlist(my.count), ncol=3, byrow=TRUE));
colnames(my.count.df) = names(my.count[[1]]);

my.count.df;

#    syllables vowels         word
# 1          4      4   A-me-ri-ca
# 2          4      5 be-auti-fu-l
# 3          3      4   spa-ci-ous
# 4          2      2       ski-es
# 5          2      2       a-mber
# 6          2      2       wa-ves
# 7          2      2       gra-in
# 8          2      2      pu-rple
# 9          3      4  mo-unta-ins
# 10         3      3    ma-je-sty

我没有意识到这是一个多么深奥的“兔子洞”，看起来很简单。


################ hackathon #######


# https://en.wikipedia.org/wiki/Gunning_fog_index
# THIS is a CLASSIFIER PROBLEM ...
# https://dev59.com/ZXRC5IYBdhLWcg3wG9Xp



# http://www.speech.cs.cmu.edu/cgi-bin/cmudict
# http://www.syllablecount.com/syllables/


  # https://enchantedlearning.com/consonantblends/index.shtml
  # start.digraphs = c("bl", "br", "ch", "cl", "cr", "dr", 
  #                   "fl", "fr", "gl", "gr", "pl", "pr",
  #                   "sc", "sh", "sk", "sl", "sm", "sn",
  #                   "sp", "st", "sw", "th", "tr", "tw",
  #                   "wh", "wr");
  # start.trigraphs = c("sch", "scr", "shr", "sph", "spl",
  #                     "spr", "squ", "str", "thr");
  # 
  # 
  # 
  # end.digraphs = c("ch","sh","th","ng","dge","tch");
  # 
  # ile
  # 
  # farmer
  # ar er
  # 
  # vowel teams ... beaver1
  # 
  # 
  # # "able"
  # # http://www.abcfastphonics.com/letter-blends/blend-cial.html
  # blends = c("augh", "ough", "tien", "ture", "tion", "cial", "cian", 
  #             "ck", "ct", "dge", "dis", "ed", "ex", "ful", 
  #             "gh", "ng", "ous", "kn", "ment", "mis", );
  # 
  # glue = c("ld", "st", "nd", "ld", "ng", "nk", 
  #           "lk", "lm", "lp", "lt", "ly", "mp", "nce", "nch", 
  #           "nse", "nt", "ph", "psy", "pt", "re", )
  # 
  # 
  # start.graphs = c("bl, br, ch, ck, cl, cr, dr, fl, fr, gh, gl, gr, ng, ph, pl, pr, qu, sc, sh, sk, sl, sm, sn, sp, st, sw, th, tr, tw, wh, wr");
  # 
  # # https://mantra4changeblog.wordpress.com/2017/05/01/consonant-digraphs/
  # digraphs.start = c("ch","sh","th","wh","ph","qu");
  # digraphs.end = c("ch","sh","th","ng","dge","tch");
  # # https://www.education.com/worksheet/article/beginning-consonant-blends/
  # blends.start = c("pl", "gr", "gl", "pr",
  #                 
  # blends.end = c("lk","nk","nt",
  # 
  # 
  # # https://sarahsnippets.com/wp-content/uploads/2019/07/ScreenShot2019-07-08at8.24.51PM-817x1024.png
  # # Monte     Mon-te
  # # Sophia    So-phi-a
  # # American  A-mer-i-can
  # 
  # n.vowels = 0;
  # for(i in 1:n.char)
  #   {
  #   my.char = word.vec[i];
  # 
  # 
  # 
  # 
  # 
  # n.syll = 0;
  # str = "";
  # 
  # previous = "C"; # consonant vs "V" vowel
  # 
  # for(i in 1:n.char)
  #   {
  #   my.char = word.vec[i];
  #   
  #   my.vowel = is.element(tolower(my.char), vowels);
  #   if(my.vowel)
  #     {
  #     n.vowels = 1 + n.vowels;
  #     if(previous == "C")
  #       {
  #       if(i == 1)
  #         {
  #         str = paste0(my.char, "-");
  #         } else {
  #                 if(n.syll > 1)
  #                   {
  #                   str = paste0(str, "-", my.char);
  #                   } else {
  #                          str = paste0(str, my.char);
  #                         }
  #                 }
  #        n.syll = 1 + n.syll;
  #        previous = "V";
  #       } 
  #     
  #   } else {
  #               str = paste0(str, my.char);
  #               previous = "C";
  #               }
  #   #
  #   }
  # 
  # 
  # 
  # 
## https://jzimba.blogspot.com/2017/07/an-algorithm-for-counting-syllables.html
# AIDE   1
# IDEA   3
# IDEAS  2
# IDEE   2
# IDE   1
# AIDA   2
# PROUSTIAN 3
# CHRISTIAN 3
# CLICHE  1
# HALIDE  2
# TELEPHONE 3
# TELEPHONY 4
# DUE   1
# IDEAL  2
# DEE   1
# UREA  3
# VACUO  3
# SEANCE  1
# SAILED  1
# RIBBED  1
# MOPED  1
# BLESSED  1
# AGED  1
# TOTED  2
# WARRED  1
# UNDERFED 2
# JADED  2
# INBRED  2
# BRED  1
# RED   1
# STATES  1
# TASTES  1
# TESTES  1
# UTILIZES  4

为了更好地衡量，这里有一个简单的金凯德可读性函数... 音节是从第一个函数返回的计数列表...

由于我的函数有点偏向于更多的音节，这将给出一个夸大的可读性分数... 目前来说这没问题... 如果目标是使文本更易读，这不是最糟糕的事情。

computeReadability = function(n.sentences, n.words, syllables=NULL)
  {
  n = length(syllables);
  n.syllables = 0;
  for(i in 1:n)
    {
    my.syllable = syllables[[i]];
    n.syllables = my.syllable$syllables + n.syllables;
    }
  # Flesch Reading Ease (FRE):
  FRE = 206.835 - 1.015 * (n.words/n.sentences) - 84.6 * (n.syllables/n.words);
  # Flesh-Kincaid Grade Level (FKGL):
  FKGL = 0.39 * (n.words/n.sentences) + 11.8 * (n.syllables/n.words) - 15.59; 
  # FKGL = -0.384236 * FRE - 20.7164 * (n.syllables/n.words) + 63.88355;
  # FKGL = -0.13948  * FRE + 0.24843 * (n.words/n.sentences) + 13.25934;
  
  list("FRE" = FRE, "FKGL" = FKGL); 
  }
-2

我曾经使用jsoup来完成这件事。这是一个样例音节解析器：

public String[] syllables(String text){
        String url = "https://www.merriam-webster.com/dictionary/" + text;
        String relHref;
        try{
            Document doc = Jsoup.connect(url).get();
            Element link = doc.getElementsByClass("word-syllables").first();
            if(link == null){return new String[]{text};}
            relHref = link.html(); 
        }catch(IOException e){
            relHref = text;
        }
        String[] syl = relHref.split("·");
        return syl;
    }
1
这怎么是一个通用的音节解析器？看起来这个代码只是在字典中查找音节。 - Nico Haase
网页内容由stack overflow 提供, 点击上面的
可以查看英文原文，
原文链接