使用Fluentd解析器解析音频代码日志

问题描述

嗨，我正在尝试从Docker容器日志文件中解析日志。 日志文件如下所示：

{"log":"[S=26853604] [BID=fb0e19:3]  (N 24959591) HandleARecordQuery - Host:sip3.pstnhub.microsoft.com is not in cache,setting timer [Time:18-09@08:30:55.728]\n","stream":"stdout","time":"2020-09-18T07:30:57.683119437Z"} {"log":"[S=26853605] [BID=fb0e19:3]  (N 24959592) DNSResolver::HandleTimerExpiredOnWaitForARecord: host:sip3.pstnhub.microsoft.com [Time:18-09@08:30:55.788]\n","time":"2020-09-18T07:30:57.683123737Z"} {"log":"[S=26853606] [BID=fb0e19:3]  (N 24959593) DNSResolver::HandleTimerExpOnWaitARecord - Host:sip3.pstnhub.microsoft.com resolved in external table [Time:18-09@08:30:55.788]\n","time":"2020-09-18T07:30:57.683127537Z"} {"log":"[S=26853607] [BID=fb0e19:3]  (N 24959594) SIPServersIPList::AddResolvedProxiesIPToList (ProxySet 1) - sip3.pstnhub.microsoft.com was resolved by DNS to 52.114.7.24 [Time:18-09@08:30:55.788]\n","time":"2020-09-18T07:30:57.683207738Z"} {"log":"[S=26853608] [BID=fb0e19:3]  (N 24959595) SIPServersIPList::UpdateList (ProxySet 1) - Update process finished [Time:18-09@08:30:55.788]\n","time":"2020-09-18T07:30:57.683236038Z"}

这是我流利的解析器配置：

<source>
  @id fluentd-containers.log
  @type tail
  path /var/log/containers/*.log
  pos_file /var/log/containers.log.pos
  tag raw.kubernetes.*
  read_from_head true
  <parse>
    @type multi_format
    <pattern>
      format json
      time_key time
      time_format %Y-%m-%dT%H:%M:%S.%NZ
    </pattern>
    <pattern>
      format /^(?<time>.+) (?<stream>stdout|stderr) [^ ]* (?<log>.*)$/
      time_format %Y-%m-%dT%H:%M:%S.%N%:z
    </pattern>
  </parse>
</source>
<filter kubernetes.var.log.containers.**count-log**.log>
  @id filter_rsyslog
  @type parser
  key_name log
  reserve_data true
  <parse>
    @type regexp
    expression /^.*?\[S=(?<skey>.[0-9]+)\].*?(\[SID=(?<sid>.*?)\] | \s)(?<message>.*?)\[Time:(?<timedata>.*?)\]$/
  </parse>
</filter>

问题在于，fluentd将多条日志消息组合到一个条目中，并且生成的json如下：

{
"log": "[S=26852255] [SID=fb0e19:3:768876]  (N 24958326) (#498)gwSession[Allocated]. Handle:00007FD174227408; Global session ID: 899236dede666591 [Time:18-09@08:26:32.000]\n","stream": "stdout","skey": "26852255","sid": "fb0e19:3:768876","message": " (N 24958326) (#498)gwSession[Allocated]. Handle:00007FD174227408; Global session ID: 899236dede666591  (N 24958335) ---- Incoming SIP Message from 52.114.148.0:5061 to SIPInterface #1 (Teams_SIPInterface) TLS TO(#1520) SocketID(362205) ----  SIP/2.0 200 OK #012FROM: <sip:11.1.0.5>;tag=1c1113388569 #012TO: <sip:11.1.0.5> #012CSEQ: 1 OPTIONS #012CALL-ID: [e-mail] #012VIA: SIP/2.0/TLS sbc.connecttoteams.com:5061;branch=z9hG4bKac2067450505 #012CONTENT-LENGTH: 0 #012ALLOW: INVITE,ACK,OPTIONS,CANCEL,BYE,NOTIFY #012SERVER: Microsoft.PSTNHub.SIPProxy v.2020.9.5.3 i.USWE2.4 #012 #012  (N 24958336) SIPLadder::FillVQMData not generated  (N 24958337) AcSIPDialog(#10663): Handling 200 OK in state DialogInitiated  (N 24958338) States: (#10663)AcSIPDialog[DialogInitiated->DialogConnected]  (N 24958339) SIPServersMngr::UpdateSetWithOnlineServer - Server 52.114.148.0 already on working servers list  (N 24958340) AcSIPDialog(#10663): Handling DIALOG_DISCONNECT_REQ in state DialogConnected  (N 24958341) States: (#10663)AcSIPDialog[DialogConnected->DialogDisconnected]  (N 24958342) SIPAppMngr::FreeDialogAPI - (#1557)  (N 24958343) States: (#10663)AcSIPDialog[Deallocated] #012 (#10663)AcSIPDialog[DialogDisconnected->DialogIdle]  (N 24958344) (#498)gwSession[Deallocated]  (N 24958345) (#518)gwSession[Allocated]. Handle:00007FD1742270E8; Global session ID: 012e10c242bc7c0b  (N 24958346) Condition Table matched on condition Index 0  (N 24958347) Classification Succeeded - Source IP Group #1 (Teams_IPGroup)  (N 24958348) States: (#1170)SBCRoutesIterator[InitialRouting->AlternativeRouting]  (N 24958349) SBC_ADMIT_DIALOGS_EV: (#1170)SBCRoutesIterator -> (#0)SBCAdmissionControlMngr  (N 24958350) CAC: Add SBC Incoming Other,IPG 1 (Teams_IPGroup): 1,SRD 0 (DefaultSRD): 1,SipIF 1 (Teams_SIPInterface): 1  (N 24958351) CAC: Add SBC Outgoing Other,SipIF 1 (Teams_SIPInterface): 1  (N 24958352) (#1170)Route found (0),Route by Address,IP Group 1 -> 1 (Teams_IPGroup -> Teams_IPGroup),Url:internal:0;" 
}

如何修改正则表达式或流利的配置，以确保消息仅包含单个日志条目的内容？

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）