Logstash多个无法解析和添加新日期

问题描述

我正在尝试以这种方式重写通过logstash发送的日志的时间戳：

filter {
    grok {
      match => { "message" => "(?<timestamp>%{DAY:day} %{MONTH:month} *%{MONTHDAY:mday} %{TIME:time} %{YEAR:year})(?<message>.*$)" }
      overwrite => [ "message" ]
      add_tag => [ "oracle_audit" ]
    }
grok {
     match => { "message" => "ACTION :\[[0-9]*\] '(?<ora_audit_action>.*)'.*DATABASE USER:\[[0-9]*\] '(?<ora_audit_dbuser>.*)'.*PRIVILEGE :\[[0-9]*\] '(?<ora_audit_priv>.*)'.*CLIENT USER:\[[0-9]*\] '(?<ora_audit_osuser>.*)'.*CLIENT TERMINAL:\[[0-9]*\] '(?<ora_audit_term>.*)'.*STATUS:\[[0-9]*\] '(?<ora_audit_status>.*)'.*DBID:\[[0-9]*\] '(?<ora_audit_dbid>.*)'.*SESSIONID:\[[0-9]*\] '(?<ora_audit_sessionid>.*)'.*USERHOST:\[[0-9]*\] '(?<ora_audit_dbhost>.*)'.*CLIENT ADDRESS:\[[0-9]*\] '(?<ora_audit_clientaddr>.*)'.*ACTION NUMBER:\[[0-9]*\] '(?<ora_audit_actionnum>.*)'" }
}
prune {
     whitelist_names => ["ora_audit_action","ora_audit_dbuser","ora_audit_dbid","ora_audit_status","ora_audit_osuser","ora_audit_priv","ora_audit_term","ora_audit_sessionid","ora_audit_dbhost","ora_audit_clientaddr","ora_audit_actionnum","host","@timestamp","@version","message"]
}

mutate {
       add_field => {
        "timestamp" => "%{year} %{month} %{mday} %{time}"
       }
  }

  date {
      locale => "en"
      match => [ "timestamp","yyyy MMM dd HH:mm:ss","yyyy MMM d HH:mm:ss","yyyy MMM  d HH:mm:ss" ]
      timezone => "UTC"
    }
}

但是它一直给我解析器错误：

{
         "ora_audit_term" => "pts/0","ora_audit_actionnum" => "3","@version" => "1","ora_audit_dbuser" => "/","ora_audit_sessionid" => "4294967295","ora_audit_action" => "SELECT STATUS FROM V$INSTANCE","tags" => [
        [0] "_dateparsefailure"
    ],"ora_audit_osuser" => "test_user","ora_audit_dbid" => "1762369616","timestamp" => "%{year} %{month} %{mday} %{time}","@timestamp" => 2020-10-05T13:43:38.238Z,"ora_audit_priv" => "SYSDBA","message" => " +00:00\nLENGTH : '296'\nACTION :[29] 'SELECT STATUS FROM V$INSTANCE'\nDATABASE USER:[1] '/'\nPRIVILEGE :[6] 'SYSDBA'\nCLIENT USER:[9] 'test_user'\nCLIENT TERMINAL:[5] 'pts/0'\nSTATUS:[1] '0'\nDBID:[10] '1762369616'\nSESSIONID:[10] '4294967295'\nUSERHOST:[21] 'testdevserver'\nCLIENT ADDRESS:[0] ''\nACTION NUMBER:[1] '3'","ora_audit_dbhost" => "testdevserver","ora_audit_status" => "0"
}

以下是示例日志：


Thu Oct  1 23:01:00 2020 +00:00
LENGTH : '296'
ACTION :[29] 'SELECT STATUS FROM V$INSTANCE'
DATABASE USER:[1] '/'
PRIVILEGE :[6] 'SYSDBA'
CLIENT USER:[9] 'test_user'
CLIENT TERMINAL:[5] 'pts/0'
STATUS:[1] '0'
DBID:[10] '1762369616'
SESSIONID:[10] '4294967295'
USERHOST:[21] 'testdevserver'
CLIENT ADDRESS:[0] ''
ACTION NUMBER:[1] '3'

我该怎么做才能正确解析时间戳，使其与日志的时间戳匹配？：（

谢谢 J

解决方法

“时间戳” =>“％{year}％{month}％{mday}％{time}”，

日期过滤器失败，因为它无法解析它。 mutate + add_field会创建该字段，因为这些字段都不存在。即使grok创建了这些字段，prune + whitelist_names也会删除它们！

logstash logstash-grok