需求
我需要一个类似于Google搜索框中建议的功能,Solr已经提供了。
结果应该如下所示:
搜索词 Alex
结果 Alexander Behling, Alexander Someone ...
搜索词 cab
结果 cable, high voltage cable, cable cutter
目标是将短语作为建议,而不是整个字段或摘录。查询应该不区分大小写,Alex应该有与alex相同的结果,但搜索结果(建议)必须具有原始大小写。
建议必须可以通过类别进行过滤,我们在一个索引中拥有多个领域的结果,并且结果应该由包含该领域的特定字段进行过滤。contextField只能在“AnalyzingInfixLookupFactory和BlendedInfixLookupFactory目前支持此功能,当由DocumentDictionaryFactory支持时”。
我尝试了三种方法
1. 方法:FreeTextLookupFactory
config (no special schema changes):
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">default</str>
<str name="lookupImpl">FreeTextLookupFactory</str>
<str name="dictionaryImpl">DocumentDictionaryFactory</str>
<str name="field">content</str>
<str name="ngrams">3</str>
<str name="separator"> </str>
<str name="suggestFreeTextAnalyzerFieldType">text_general</str>
</lst>
</searchComponent>
<requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy" >
<lst name="defaults">
<str name="suggest">true</str>
<str name="suggest.count">10</str>
<str name="suggest.dictionary">default</str>
<str name="echoParams">explicit</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
这种方法相当可行,但只返回单个词。
检索关键词 Alex
结果 Alexander, Alexandra ...
优点是索引速度非常快。我尝试将其与ShingleFilter结合使用,但这并没有奏效,可能是因为ShingleFilter已经是FreeTextLookupFactory的一部分了。由于FreeTextLookupFactory不支持类别,所以无法使用它们。
2. 方法:使用自定义字段的BlendedInfixLookupFactory
schema:
<field name="suggest_field" type="text_suggest" indexed="true" stored="true" multiValued="true"/>
<field name="site" type="string" stored="true" indexed="true"/>
<copyField source="content" dest="suggest_field"/>
<fieldType name="text_suggest" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
<!--filter class="solr.LowerCaseFilterFactory"/-->
<filter class="solr.TrimFilterFactory"/>
<filter class="solr.ShingleFilterFactory"
minShingleSize="2"
maxShingleSize="4"
outputUnigrams="true"
outputUnigramsIfNoShingles="true"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.KeywordTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
</fieldType>
config:
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">default</str>
<str name="lookupImpl">BlendedInfixLookupFactory</str>
<str name="blenderType">position_linear</str>
<str name="dictionaryimpl">DocumentDictionaryFactory</str>
<str name="field">suggest_field</str>
<str name="weightField">weight</str>
<str name="suggestAnalyzerFieldType">text_suggest</str>
<str name="queryAnalyzerFieldType">phrase_suggest</str>
<str name="indexPath">suggest</str>
<str name="buildOnStartup">false</str>
<str name="buildOnCommit">false</str>
<bool name="exactMatchFirst">true</bool>
<str name="contextField">site</str>
</lst>
</searchComponent>
<requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy" >
<lst name="defaults">
<str name="suggest">true</str>
<str name="suggest.count">10</str>
<str name="suggest.dictionary">default</str>
<str name="echoParams">explicit</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>FreeTextLookupFactory
第二种方法会导致奇怪的行为:
搜索词 Alex 或 alex
结果 没有任何结果...
搜索词 cab
结果 电缆,电缆,电缆附件,电力电缆...
使用相同的字段,某些查询没有搜索结果。索引速度已经超过了 10000 条目的 >12h。由于使用了 BlendedInfixLookupFactory 和 DocumentDictionaryFactory,因此应支持类别。但是,在查询中使用类别时,例如:
http://localhost:8983/solr/magnolia/suggest?wt=json&suggest=true&suggest.q=nym&suggest.cfq=com
,结果为空。字段“site”在索引中多次包含值“com”。第 3 种方法是使用 BlendedInfixLookupFactory、HighFrequencyDictionaryFactory 和自定义字段。
schema:
<field name="suggest_field" type="text_shingle" indexed="true" stored="true" multiValued="true"/>
...
<copyField source="_text_" dest="suggest_field"/>
...
<fieldType name="text_shingle" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<charFilter class="solr.HTMLStripCharFilterFactory"/>
<filter class="solr.TrimFilterFactory"/>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords_suggestions.txt" format="snowball" />
<!--filter class="solr.EdgeNGramFilterFactory" minGramSize="4" maxGramSize="15"/-->
<filter class="solr.ShingleFilterFactory" minShingleSize="2" maxShingleSize="4" outputUnigrams="false" outputUnigramsIfNoShingles="true" fillerToken=""/>
</analyzer>
</fieldType>
<!-- marc johnen : used for autocomplete-->
<fieldType name="text_suggest" class="solr.TextField" positionIncrementGap="100">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.TrimFilterFactory"/>
</analyzer>
</fieldType>
config:
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">default</str>
<str name="lookupImpl">BlendedInfixLookupFactory</str>
<str name="dictionaryImpl">HighFrequencyDictionaryFactory</str>
<str name="field">suggest_field</str>
<str name="suggestAnalyzerFieldType">text_suggest</str>
<str name="minPrefixChars">2</str>
<str name="exactMatchFirst">true</str>
<str name="buildOnStartup">false</str>
<str name="buildOnCommit">true</str>
<str name="highlight">false</str>
</lst>
</searchComponent>
<requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy" >
<lst name="defaults">
<str name="suggest">true</str>
<str name="suggest.count">10</str>
<str name="suggest.dictionary">default</str>
<str name="echoParams">explicit</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
这种方法的结果相当不错,基本上按照规定进行,但由于某些关键词在开头或结尾处有空格,例如“power cable”和“power cable”,因此存在一些重复短语。除此以外,非常好。
搜索词 Alex
结果 Alexander Behling, Alexander Someone ...
搜索词 cab
结果 电缆,高压电缆,电缆切割器
对于小于10000个文档,索引很容易花费一天时间。主要问题是由于HighFrequencyDictionaryFactory不支持类别。
查询
我使用的查询如下:
http://localhost:8983/solr/magnolia/suggest?wt=json&suggest=true&suggest.q=cab
在配置文件中添加<str name="contextField">site</str>
用于类别,并在适用时将&suggest.cfq=com
添加到查询中。