问题描述
这是我第一次使用grok过滤器，我想从日志中提取有用的键。我无法创建grok模式。
我有access.log文件，并使用logstash将日志数据放入elsticsearch中。
示例日志
September 6th 2020,10:30:37:759 am [f657a07c-8c62-46ee-78a3-e7c2a04328b0] info: authentication successful...
September 6th 2020,10:30:38:179 am [f657a07c-8c62-46ee-78a3-e7c2a04328b0] info: external apiresponse time: 3 ms
September 6th 2020,10:30:49:200 am [f657a07c-8c62-46ee-78a3-e7c2a04328b0] info: {"path":"/route1","originalUrl":"/media/auth/route1?id=601&userId=9e2681-a188-4978-94b0-04c1d50ad056","timestamp":1599368449200,"uuid":"f657a07c-8c62-46ee-88a3-e7c3a04328b0","responseCode":"200","responseTime":11455,"externalCalls":[{"type":"apis","status":"success","tiMetaken":2167},{"type":"pdf-generate-time","tiMetaken":5302},{"type":"pdf-upload-s3","tiMetaken":1146}]}
我只想处理{“ path”：........}这个对象，以进行弹性搜索并提取所有键作为Elastic中的字段。
我的logstash conf文件-
{
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:log-level} \[%{DATA:class}\]:%{GREEDYDATA:message}" }
}
}
解决方法
如果日志格式始终相同，则可以使用以下模式：info: %{GREEDYDATA:text}$
在您的示例中，它将使用以下值创建一个名为text的字段：
authentication successful...
external apiresponse time: 3 ms
{"path":"/route1","originalUrl":"/media/auth/route1?id=601&userId=9e2681-a188-4978-94b0-04c1d50ad056","timestamp":1599368449200,"uuid":"f657a07c-8c62-46ee-88a3-e7c3a04328b0","responseCode":"200","responseTime":11455,"externalCalls":[{"type":"apis","status":"success","timeTaken":2167},{"type":"pdf-generate-time","timeTaken":5302},{"type":"pdf-upload-s3","timeTaken":1146}]}
然后您可以使用json过滤器对其进行解析。