Elasticsearch:父子关系不一致

问题描述

我有一个网页,其中根据用户对问题的回答显示了汽车列表,在导航至汽车页面之前,他/她应回答这些问题。换句话说,我有一个规则,指示应该/不应该为用户显示哪些汽车。例如:使宝马汽车仅在用户回答“您最喜欢的颜色是什么?”的问题时使用带有“黑色”或“您几岁?”带有选项“ 18”。因此,当用户导航到汽车页面时,应用程序逻辑通过将用户选择的答案与规则中配置的答案相匹配来评估已配置的汽车规则。

为了允许用户在该页面上进行快速搜索,将汽车及其规则放到了由3个节点version 6.2.4组成的Elasticsearch中 汽车及其规则之间的关系定义为join type

"carRulesRelationship": {
                        "type": "join","eager_global_ordinals": true,"relations": {
                            “car”: [
                                “carAvailability"                             
                            ]
                        }
  }

汽车文档样本:

"_type": "doc","_id": "233492","_source": {
        "id0": 233492,"name": "BMW","carRulesRelationship": "car","status": "active","updatedate23": "2020-10-01T01:00:16.280Z","@timestamp": "2020-10-19T11:19:29.865Z"
    }

规则示例:

{
    "_type": "doc","_id": "233492-18474","_routing": "233492","found": true,"_source": {
        "ruleType": "carAvailability","ruleUpdatedDate": "2020-10-07T00:09:42.993Z","carAvailability": {
            "visible": true,"userMessage": "You are no longer eligible for this car."
        },"ruleName": "BMW rule","ruleConditionalOperator": "or","requiredMatches": 4,"ruleCreatedDate": "2020-08-19T15:42:31.327Z","questionDetailsConditionalRules": [
            {
                "questionDetailId": 13994034
            },{
                "questionDetailId": 13994031
            },{
                "questionDetailId": 13994035
            },{
                "questionDetailId": 13994024
            }
        ],"ruleId": 18474,"carRulesRelationship": {
            "name": "carAvailability","parent": "233492"
        }
    }
}

基于用户选择的答案ID的搜索查询,仅返回用户应该看到的汽车:

{
  "from": 0,"size": 100,"query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "query": "*","fields": [
              "searchfield^1.0"
            ],"type": "best_fields","default_operator": "and","max_determinized_states": 10000,"enable_position_increments": true,"fuzziness": "0","fuzzy_prefix_length": 0,"fuzzy_max_expansions": 1,"phrase_slop": 0,"escape": false,"auto_generate_synonyms_phrase_query": false,"fuzzy_transpositions": false,"boost": 1.0
          }
        }
      ],"filter": [
        {
          "terms": {
            "status": [
              "active","closed"
            ],"must_not": [
        {
          "has_child": {
            "query": {
              "bool": {
                "filter": [
                  {
                    "term": {
                      "carAvailability.visible": {
                        "value": false,"boost": 1.0
                      }
                    }
                  },{
                    "term": {
                      "ruleConditionalOperator": {
                        "value": "and",{
                    "terms_set": {
                      "questionDetailsConditionalRules.questionDetailId": {
                        "terms": [
                          13994021,13994032,13994042,13994040,13993960,13993983,13994024
                        ],"minimum_should_match_field": "requiredMatches","boost": 1.0
                      }
                    }
                  }
                ],"adjust_pure_negative": true,"boost": 1.0
              }
            },"type": "carAvailability","score_mode": "none","min_children": 0,"max_children": 2147483647,"ignore_unmapped": false,"boost": 1.0
          }
        },{
          "has_child": {
            "query": {
              "bool": {
                "filter": [
                  {
                    "term": {
                      "carAvailability.visible": {
                        "value": false,{
                    "term": {
                      "ruleConditionalOperator": {
                        "value": "or",{
                    "terms": {
                      "questionDetailsConditionalRules.questionDetailId": [
                        13994021,13994024
                      ],"boost": 1.0
                    }
                  }
                ],{
          "has_child": {
            "query": {
              "bool": {
                "filter": [
                  {
                    "term": {
                      "carAvailability.visible": {
                        "value": true,{
                    "bool": {
                      "must_not": [
                        {
                          "terms_set": {
                            "questionDetailsConditionalRules.questionDetailId": {
                              "terms": [13994021,13994024],"boost": 1.0
                            }
                          }
                        }
                      ],{
                    "bool": {
                      "must_not": [
                        {
                          "terms": {
                            "questionDetailsConditionalRules.questionDetailId": [13994021,"boost": 1.0
                          }
                        }
                      ],"boost": 1.0
    }
  }
}

汽车和规则数据更新过程: logstash 在ES中刷新了汽车及其规则,该日志从MS SQL DB获取数据并将其输出到ES。 配置了2个管道:

  1. 汽车管道-每3分钟运行一次,并对ES中的所有汽车重新编制索引(3个索引对应3个环境-QA,STG,PROD。每个索引的汽车数量为177K),无论是否更新。
  2. 规则管道-每2分钟运行一次,并且仅提取自上次logstash循环迭代以来的新的/更新的DB记录。

问题::一些用户告知他们看到了自己不应看到的汽车,因为他们没有选择汽车规则要求的答案,例如:the car rule requires answer with Id 13994028 to be visible for user.在上面运行搜索查询对于此类用户,始终会返回正确的汽车,而不会出错。偶尔只有少数用户仍然会遇到此问题,而我仍然无法理解根本原因。

问题:在 logstash 重新索引汽车文档时,汽车和规则文档之间的父/子关系是否可能暂时中断。那么用户进入汽车文档重新索引时间窗口,就能看到错误的汽车吗?

解决方法

暂无找到可以解决该程序问题的有效方法,小编努力寻找整理中!

如果你已经找到好的解决方法,欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@)