问题描述
大家好,我是 Elasticsearch 的新手,我有一张名为 'american_football_sacks_against_stats' 的表。它由三列组成。
id | sacks_against_total | sacks_against_yards |
---|---|---|
1 | 12 | 5 |
2 | 15 | 3 |
... | ... | ... |
问题是 sacks_against_total 和 sacks_against_yards 不是作为整数/长/浮点数“导入”的,而是作为文本字段和关键字字段。如何将它们转换为数字?
我试过了,但它不起作用:
mutate {
convert => {
"id" => "integer"
"sacks_against_total" => "integer"
"sacks_against_yards" => "integer"
}
}
这是我的 logstash.conf 文件:
input {
jdbc {
jdbc_connection_string => "jdbc:postgresql://localhost:5432/sportsdb"
jdbc_user => "user"
jdbc_password => "password"
jdbc_driver_class => "org.postgresql.Driver"
schedule => "*/5 * * * *"
statement => "SELECT * FROM public.american_football_sacks_against_stats"
jdbc_paging_enabled => "true"
jdbc_page_size => "300"
}
}
filter {
mutate {
convert => {
"id" => "integer"
"sacks_against_total" => "integer"
"sacks_against_yards" => "integer"
}
}
}
output {
stdout { codec => "json" }
elasticsearch {
hosts => "http://localhost:9200"
index => "sportsdb"
doc_as_upsert => true #
}
}
解决方法
Elasticsearch 不提供更改现有字段类型的功能。
您可以在这里阅读一些选项: https://medium.com/@max.borysov/change-field-type-in-elasticsearch-index-2d11bb366517
,这是我一直在寻找的解决方案: https://linuxhint.com/elasticsearch-reindex-change-field-type/
首先从索引创建输入并更改类型。
PUT _ingest/pipeline/convert_pipeline
{
“description”: “converts the field sacks_against_total,sacks_against_yards fields to a long from string”,"processors" : [
{
"convert" : {
"field" : "sacks_against_yards","type": "long"
}
},{
"convert" : {
"field" : "sacks_against_total","type": "long"
}
}
]
}
或在 cURL 中:
curl -XPUT "http://localhost:9200/_ingest/pipeline/convert_pipeline" -H 'Content-Type: application/json' -d'{ "description": "converts the sacks_against_total field to a long from string","processors" : [ { "convert" : { "field" : "sacks_against_total","type": "long" } },{"convert" : { "field" : "sacks_against_yards","type": "long" } } ]}'
然后将索引重新索引为另一个索引
POST _reindex
{
“source”: {
"index": "american_football_sacks_against_stats"
},"dest": {
"index": "american_football_sacks_against_stats_withLong","pipeline": "convert_pipeline"
}
}
或在 cURL 中:
curl -XPOST "http://localhost:9200/_reindex" -H 'Content-Type: application/json' -d'{ "source": { "index": "sportsdb" },"dest": { "index": "sportsdb_finish","pipeline": "convert_pipeline" }}'