Elasicsearch 将 NGram 与简单查询字符串查询混合

问题描述

目前，我正在使用 Ngram 标记器来进行员工的部分匹配。

我可以匹配全名、电子邮件地址和员工编号

我当前的设置如下：

"tokenizer": {
  "my_tokenizer": {
    "type": "ngram","min_gram": 3,"max_gram": 3,"token_chars": [
      "letter","digit"
    ]
  }
}

我面临的问题是 Employee Number 可以是 1 个字符长，并且由于 min_gram 和 max_gram，我永远不能匹配。我也无法将 min_gram 设为 1，因为结果看起来不正确。

所以我尝试将 Ngram 与标准分词器混合使用，而不是在 Multimatch 搜索中进行，而是使用 simple_query_string。

这似乎也部分起作用。

我的问题是如何部分匹配所有 3 个字段，请记住员工编号可以是 1 或 2 个字符长。如果我在单词或数字周围使用半引号，则完全匹配

在下面的示例中，如何搜索 11 并返回文档 4 和 5？ 另外，如果我必须搜索部分匹配的 706，我希望文档 2 返回，但是如果我必须使用“7061”进行搜索，我将只返回文档 2

完整代码

PUT index
{
  "settings": {
    "analysis": {
      "analyzer": {
        "english_exact": {
          "tokenizer": "standard","filter": [
            "lowercase"
          ]
        },"my_analyzer": {
            "filter": [
              "lowercase","asciifolding"
            ],"tokenizer": "my_tokenizer"
          }
      },"tokenizer": {
          "my_tokenizer": {
            "type": "ngram","token_chars": [
              "letter","digit"
            ]
          }
        },"normalizer": {
          "lowersort": {
            "type": "custom","filter": [
              "lowercase"
            ]
          }
        }
    }
  },"mappings": {
    "properties": {
      "number": {
        "type": "text","analyzer": "english","fields": {
          "exact": {
            "type": "text","analyzer": "english_exact"
          }
        }
      },"fullName": {
        "type": "text","fields": {
          "ngram": {
            "type": "text","analyzer": "my_analyzer"
          }
        },"analyzer": "standard"
      }
    }
  }
}
PUT index/_doc/1
{
  "number" : 1,"fullName": "Brenda eaton"
}

PUT index/_doc/2
{
  "number" : 7061,"fullName": "Bruce wayne"
}

PUT index/_doc/3
{
  "number" : 23,"fullName": "Bruce Banner"
}

PUT index/_doc/4
{
  "number" : 111,"fullName": "Cat woman"
}

PUT index/_doc/5
{
  "number" : 1112,"fullName": "0723568521"
}

GET index/_search
{
  "query": {
    "simple_query_string": {
      "fields": [ "fullName.ngram","number.exact"],"query": "11"
    }
  }
}

解决方法

您需要更改number.exact字段的分析器并减少min_gram 计数为2。修改索引映射如下图

添加一个工作示例

索引映射：

    {
  "settings": {
    "analysis": {
      "analyzer": {
        "english_exact": {
          "tokenizer": "standard","filter": [
            "lowercase"
          ]
        },"my_analyzer": {
          "filter": [
            "lowercase","asciifolding"
          ],"tokenizer": "my_tokenizer"
        }
      },"tokenizer": {
        "my_tokenizer": {
          "type": "ngram","min_gram": 2,"max_gram": 3,"token_chars": [
            "letter","digit"
          ]
        }
      },"normalizer": {
        "lowersort": {
          "type": "custom","filter": [
            "lowercase"
          ]
        }
      }
    }
  },"mappings": {
    "properties": {
      "number": {
        "type": "keyword",// note this
        "fields": {
          "exact": {
            "type": "text","analyzer": "my_analyzer"
          }
        }
      },"fullName": {
        "type": "text","fields": {
          "ngram": {
            "type": "text","analyzer": "my_analyzer"
          }
        },"analyzer": "standard"
      }
    }
  }
}

搜索查询：

{
  "query": {
    "simple_query_string": {
      "fields": [ "fullName.ngram","number.exact"],"query": "11"
    }
  }
}

搜索结果：

"hits": [
      {
        "_index": "66311552","_type": "_doc","_id": "4","_score": 0.9929736,"_source": {
          "number": 111,"fullName": "Cat woman"
        }
      },{
        "_index": "66311552","_id": "5","_score": 0.8505551,"_source": {
          "number": 1112,"fullName": "0723568521"
        }
      }
    ]

更新 1：

如果只需要搜索1，将number字段的数据类型由text类型修改为keyword类型，如上图索引映射.

搜索查询：

{
  "query": {
    "simple_query_string": {
      "fields": [ "fullName.ngram","number.exact","number"],"query": "1"
    }
  }
}

搜索结果将是

"hits": [
      {
        "_index": "66311552","_id": "1","_score": 1.3862942,"_source": {
          "number": 1,"fullName": "Brenda eaton"
        }
      }
    ]

更新 2：

您可以对 fullName 字段和 number 字段使用两个带有 n-gram 分词器的单独分析器。使用以下索引映射进行修改：

{
  "settings": {
    "analysis": {
      "analyzer": {
        "english_exact": {
          "tokenizer": "standard","name_analyzer": {
          "filter": [
            "lowercase","tokenizer": "name_tokenizer"
        },"number_analyzer": {
          "filter": [
            "lowercase","tokenizer": "number_tokenizer"
        }
      },"tokenizer": {
        "name_tokenizer": {
          "type": "ngram","min_gram": 3,"digit"
          ]
        },"number_tokenizer": {
          "type": "ngram","fields": {
          "exact": {
            "type": "text","analyzer": "number_analyzer"
          }
        }
      },"analyzer": "name_analyzer"
          }
        },"analyzer": "standard"
      }
    }
  }
}