logo标签列表

ElasticSearch非法参数异常

elasticsearchillegalargumentexception
3
我是在Ubuntu 16.04上使用最新版本的Elasticsearch，现在在将数据放入其中遇到了一点问题。
以下是我的JSON文档（相关部分）：
{   "products" : {
    "232CDFDW89ENUXRB" : {
        "sku" : "232CDFDW89ENUXRB",
        "productFamily" : "Compute Instance",
        "attributes" : {
            "servicecode" : "AmazonEC2",
            "location" : "US East (N. Virginia)",
            "locationType" : "AWS Region",
            "instanceType" : "d2.8xlarge",
            "currentGeneration" : "Yes",
            "instanceFamily" : "Storage optimized",
            "vcpu" : "36",
            "physicalProcessor" : "Intel Xeon E5-2676v3 (Haswell)",
            "clockSpeed" : "2.4 GHz",
            "memory" : "244 GiB",
            "storage" : "24 x 2000 HDD",
            "networkPerformance" : "10 Gigabit",
            "processorArchitecture" : "64-bit",
            "tenancy" : "Host",
            "operatingSystem" : "Linux",
            "licenseModel" : "No License required",
            "usagetype" : "HostBoxUsage:d2.8xlarge",
            "operation" : "RunInstances",
            "enhancedNetworkingSupported" : "Yes",
            "preInstalledSw" : "NA",
            "processorFeatures" : "Intel AVX; Intel AVX2; Intel Turbo" }
        }
    }   
}

当我尝试"PUT http://localhost:9200/aws"时，这里是来自ES的返回响应

{ "error": {
"root_cause": [
  {
    "type": "illegal_argument_exception",
    "reason": "unknown setting [index.products.232CDFDW89ENUXRB.attributes.clockSpeed] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"
  }
],
"type": "illegal_argument_exception",
"reason": "unknown setting [index.products.232CDFDW89ENUXRB.attributes.clockSpeed] please check that any required plugins are installed, or check the breaking changes documentation for removed settings" }, "status": 400 }

似乎 ES 认为 "clockSpeed" 是某种设置...？ 我希望使用动态映射来加速这个过程，而不是先映射整个文档，然后再将其导入 ES。
有什么建议吗？
2个回答
2
问题在于你在通过PUT http://localhost:9200/aws命令对文档进行索引时，缺失了文档类型（document type）和文档ID（document id）。
正确的文档索引方式为：
POST my-index/my-type/my-id-1
{
  "name": "kibana"
}

即您需要提供文档类型(这里是my-type)和文档ID(这里是my-id-1)。请注意，文档ID在此处是可选的，因此如果您不提供，则elasticsearch会为您创建一个字母数字ID。

索引文档的另外几种方式：

POST my-index/my-type
{
  "name": "kibana"
}

//if you want to index document through PUT then you must provide document id
PUT my-index/my-type/my-id-1
{
  "name": "kibana"
}

注意：如果自动索引创建被禁用，则必须在索引文档之前创建索引。

如果上面的答案解决了您的问题，那么您可以将其接受为答案并关闭问题。 - avr
0

在干净的映射下，XPOST 在 Elasticsearch 5.1.1 上完美运行。

$ curl -XPOST localhost:9200/productsapp/productdocs -d '
{   "products" : {
     "sku1" : {
         "sku" : "SKU-Name",
         "productFamily" : "Compute Instance",
         "attributes" : {
             "servicecode" : "AmazonEC2",
             "location" : "US East (N. Virginia)",
             "locationType" : "AWS Region",
             "instanceType" : "d2.8xlarge",
             "currentGeneration" : "Yes",
             "instanceFamily" : "Storage optimized",
             "vcpu" : "36",
             "physicalProcessor" : "Intel Xeon E5-2676v3 (Haswell)",
             "clockSpeed" : "2.4 GHz",
             "memory" : "244 GiB",
             "storage" : "24 x 2000 HDD",
             "networkPerformance" : "10 Gigabit",
             "processorArchitecture" : "64-bit",
             "tenancy" : "Host",
             "operatingSystem" : "Linux",
             "licenseModel" : "No License required",
             "usagetype" : "HostBoxUsage:d2.8xlarge",
             "operation" : "RunInstances",
             "enhancedNetworkingSupported" : "Yes",
             "preInstalledSw" : "NA",
             "processorFeatures" : "Intel AVX; Intel AVX2; Intel Turbo" }
         }
     }   
 }'
{"_index":"productsapp","_type":"productdocs","_id":"AVuhXdYYUiSguAb0FsSX","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"created":true}

获取已插入的文档

curl -XGET localhost:9200/productsapp/productdocs/_search
{"took":11,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max_score":1.0,"hits":[{"_index":"productsapp","_type":"productdocs","_id":"AVuhXdYYUiSguAb0FsSX","_score":1.0,"_source":{   "products" : {
    "sku1" : {
        "sku" : "SKU-Name",
        "productFamily" : "Compute Instance",
        "attributes" : {
            "servicecode" : "AmazonEC2",
            "location" : "US East (N. Virginia)",
            "locationType" : "AWS Region",
            "instanceType" : "d2.8xlarge",
            "currentGeneration" : "Yes",
            "instanceFamily" : "Storage optimized",
            "vcpu" : "36",
            "physicalProcessor" : "Intel Xeon E5-2676v3 (Haswell)",
            "clockSpeed" : "2.4 GHz",
            "memory" : "244 GiB",
            "storage" : "24 x 2000 HDD",
            "networkPerformance" : "10 Gigabit",
            "processorArchitecture" : "64-bit",
            "tenancy" : "Host",
            "operatingSystem" : "Linux",
            "licenseModel" : "No License required",
            "usagetype" : "HostBoxUsage:d2.8xlarge",
            "operation" : "RunInstances",
            "enhancedNetworkingSupported" : "Yes",
            "preInstalledSw" : "NA",
            "processorFeatures" : "Intel AVX; Intel AVX2; Intel Turbo" }
        }
    }   
}}]}}

它创建的映射如下，其中 clockSpeed 为 text 类型。
curl -XGET localhost:9200/productsapp/productdocs/_mapping?pretty=true
{
  "productsapp" : {
    "mappings" : {
      "productdocs" : {
        "properties" : {
          "products" : {
            "properties" : {
              "232CDFDW89ENUXRB" : {
                "properties" : {
                  "attributes" : {
                    "properties" : {
                      "clockSpeed" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "currentGeneration" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "enhancedNetworkingSupported" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "instanceFamily" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "instanceType" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "licenseModel" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "location" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "locationType" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "memory" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "networkPerformance" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "operatingSystem" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "operation" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "physicalProcessor" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "preInstalledSw" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "processorArchitecture" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "processorFeatures" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "servicecode" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "storage" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "tenancy" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "usagetype" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      },
                      "vcpu" : {
                        "type" : "text",
                        "fields" : {
                          "keyword" : {
                            "type" : "keyword",
                            "ignore_above" : 256
                          }
                        }
                      }
                    }
                  },
                  "productFamily" : {
                    "type" : "text",
                    "fields" : {
                      "keyword" : {
                        "type" : "keyword",
                        "ignore_above" : 256
                      }
                    }
                  },
                  "sku" : {
                    "type" : "text",
                    "fields" : {
                      "keyword" : {
                        "type" : "keyword",
                        "ignore_above" : 256
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

你能检查一下attributes.clockSpeed的映射，确保它没有出错吗？

如果你想更新文档，请在第一个文档的id（即AVuhXdYYUiSguAb0FsSX）上执行XPUT操作。

在下面的示例中，我将sku字段更新为"sku name updated"。

curl -XPUT localhost:9200/productsapp/productdocs/AVuhXdYYUiSguAb0FsSX -d '
{
          "products" : {
            "sku1" : {
              "sku" : "sku name updated",
              "productFamily" : "Compute Instance",
              "attributes" : {
                "servicecode" : "AmazonEC2",
                "location" : "US East (N. Virginia)",
                "locationType" : "AWS Region",
                "instanceType" : "d2.8xlarge",
                "currentGeneration" : "Yes",
                "instanceFamily" : "Storage optimized",
                "vcpu" : "36",
                "physicalProcessor" : "Intel Xeon E5-2676v3 (Haswell)",
                "clockSpeed" : "2.4 GHz",
                "memory" : "244 GiB",
                "storage" : "24 x 2000 HDD",
                "networkPerformance" : "10 Gigabit",
                "processorArchitecture" : "64-bit",
                "tenancy" : "Host",
                "operatingSystem" : "Linux",
                "licenseModel" : "No License required",
                "usagetype" : "HostBoxUsage:d2.8xlarge",
                "operation" : "RunInstances",
                "enhancedNetworkingSupported" : "Yes",
                "preInstalledSw" : "NA",
                "processorFeatures" : "Intel AVX; Intel AVX2; Intel Turbo"
              }
            }
          }}'
{"_index":"productsapp","_type":"productdocs","_id":"AVu5OLfHPw6Pv_3O38-V","_version":2,"result":"updated","_shards":{"total":2,"successful":1,"failed":0},"created":false}
同样的，只是指定索引：curl -XPOST http://localhost:9200/aws/ -d '{ ~~ }' - Emanuele Leopardi
我成功地将所有数据放入Elasticsearch中，但它为每个文档创建了一个子字段，因为它们具有不同的“sku”。我应该如何处理才能“合并”这些信息？我不关心sku的具体名称。是否可以更改它，比如改成“VM”？我应该更改映射吗？ - Emanuele Leopardi
我不太明白你的意思。你是说你为 sku1 插入了 doc1，然后又插入了具有不同 attributes 的 doc2，并且你想要将它们合并吗？ - prayagupa
不，它不会自动拆分巨大的文档，它怎么知道它们是不同的 SKU？你必须进行一些调整。您是通过CLI使用curl -XPOST还是使用像scala、python等的API？如果您正在使用curl，请执行curl -XPOST localhost:9200/_bulk --data-binary "@your_gian_json_file.json"。此外，json文件需要具有此格式，如此文档中所述-https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html - prayagupa
或者使用像Scala这样的API，您必须创建JSON对象，并迭代每个SKU并逐个插入。或者仍然可以使用批量API，但您必须让它知道每行是一个文档。 - prayagupa
显示剩余5条评论
网页内容由stack overflow 提供, 点击上面的
可以查看英文原文，
原文链接