问题描述
在 kylin Insight 中执行查询时扫描 10 439 900 行后获取协处理器超时。
我目前每年处理的数据约为 1.5 亿。
版本
- kylin-3.0.1
- hadoop-2.7.0
- hbase-1.1.2
在 kylin.properties 中的道具下方添加
kylin.query.scan.threshold=5000000000
kylin.storage.hbase.coprocessor-mem-gb=20
kylin.storage.partition.max-scan-bytes=0
异常跟踪
在执行 sql 时扫描 10439900 行后协处理器超时："select sum(gwp) gwp,sum(nwp) nwp,sum(gwp_od) gwp_od,SUM(gwp_tp) gwp_tp,sum(nwp_od) nwp_od,_sumtn(,sum(discount_gwp_od) discount_gwp_od,sum(discount_nwp_od) discount_nwp_od from ( select sum(INS_COVERAGE_PREMIUM) gwp,sum(ins_nwp) nwp,sum(case when INS_COVERAGE_CATEGORY PR = 'OD' then end_case INS_COVERAGE_CATEGORY PR = 'OD' 时,INS_OVER INS_OVER ('TP','PA') then INS_COVERAGE_PREMIUM else 0 end) gwp_tp,sum(case when INS_COVERAGE_CATEGORY = 'OD' then INS_NWP else 0 end) nwp_od,sum(case when INS_COVERAGE_CATEGORY in ('TP','PA') then INS_NWP else 0 end) nwp_tp,'PA') and INS_COVERAGE_TYPE = 'DC' then INS_COVERAGE_PREMIUM else 0 end) discount_gwp_od,'PA') and INS_COVERAGE_TYPE = 'DC' ) and INS_COVERAGE_TYPE = 'DC' then INS_NWP else 0 end) discount_nwp_od from RSA_KPI_FACT_INS_NEW ins LEFT JOIN KPI_FIN_YEAR_MASTER as KPI_FIN_ YEAR_MASTER ON ins.FINANCIAL_YEAR = KPI_FIN_YEAR_MASTER.FIN_YEAR LEFT JOIN KPI_FIN_MONTH_MASTER作为KPI_FIN_MONTH_MASTER ON ins.EFF_FIN_YEAR_MONTH = KPI_FIN_MONTH_MASTER.FIN_MONTH LEFT JOIN KPI_PRODUCT_MASTER作为KPI_PRODUCT_MASTER ON ins.PRODUCT_CODE = KPI_PRODUCT_MASTER.PRODUCT_CODE LEFT JOIN KPI_BRANCH_MASTER作为KPI_BRANCH_MASTER ON ins.BRANCH_CODE = KPI_BRANCH_MASTER.BRANCH_CODE LEFT JOIN KPI_CAMPAIGN_MASTER作为KPI_CAMPAIGN_MASTER ON ins.CAMPAIN_CODE = KPI_CAMPAIGN_MASTER.CAMPAIGN_CODE LEFT JOIN KPI_OA_MASTER_NW作为KPI_OA_MASTER_NW ON ins.OA_CODE = KPI_OA_MASTER_NW.OA_CODE LEFT JOIN KPI_MODEL_MASTER_NW作为KPI_MODEL_MASTER_NW ON ins.MAKE = KPI_MODEL_MASTER_NW.MAKE AND ins.MODELCODE = KPI_MODEL_MASTER_NW.MODEL_CODE LEFT JOIN KPI_SUB_CHANNEL_MASTER_NW作为KPI_SUB_CHANNEL_MASTER_NW ON ins.CHANNEL = KPI_SUB_CHANNEL_MASTER_NW.CHANNEL_NAME AND ins.SUB_CHANNEL = KPI_SUB_CHANNEL_MASTER_NW.SUB_CHANNEL LEFT JOIN KPI_BUSInesS_TYPE_MASTER 作为 KPI_BUSInesS_TYPE_MASTER ON ins = KPI_BUSInesS_TYPE_MASTER ON ins. nesS_TYPE_MASTER.BUSInesS_TYPE LEFT JOIN KPI_POLICY_CATEGORY_MASTER_NW作为KPI_POLICY_CATEGORY_MASTER_NW ON ins.POLICY_CATEGORY = KPI_POLICY_CATEGORY_MASTER_NW.POLICY_CATEGORY WHERE（ins.FIN_DATE> = '2018年4月1日' AND ins.FIN_DATE
解决方法
暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！
如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。
小编邮箱:dio#foxmail.com (将#修改为@）