问题描述
我尝试解析自定义日志。 Loki不接受经过分析的时间戳记原因：
level=error ts=2020-09-18T07:17:52.655628313Z msg="final error sending batch" status=400 error="server returned HTTP status 400 Bad Request (400): entry for stream has timestamp too new: 2020-09-18 10:01:52.935 +0000 UTC"
但是有时间的旧日志，例如：2020-09-09 10：01：52.935甚至2020-09-17 10：01：52.935 loki正常解析
我的日志：
2020-09-18 10:01:52.935;message
重现行为的步骤： grafana / loki：最新 grafana / promtail：最新
我的promtail配置：
...
scrape_configs:
- job_name: myjob
pipeline_stages:
- regex:
expression: '^(?P<timestamp>[\d-\s:.]{20,30});(?P<message>.*)'
- timestamp:
source: timestamp
format: 2006-01-02 15:04:05.999
static_configs:
- targets:
- localhost
labels:
job: myjob
_path_: /logs/*log
我的loki配置：
auth_enabled: false
server:
http_listen_port: 3100
ingester:
lifecycler:
address: 127.0.0.1
ring:
kvstore:
store: inmemory
replication_factor: 1
final_sleep: 0s
chunk_idle_period: 5m
chunk_retain_period: 30s
max_transfer_retries: 0
schema_config:
configs:
- from: 2018-04-15
store: boltdb
object_store: filesystem
schema: v11
index:
prefix: index_
period: 792h
storage_config:
boltdb:
directory: /tmp/loki/index
filesystem:
directory: /tmp/loki/chunks
limits_config:
enforce_metric_name: false
reject_old_samples: false
chunk_store_config:
max_look_back_period: 0s
table_manager:
retention_deletes_enabled: true
retention_period: 792h
解决方法
在 promtail 管道阶段尝试这种格式：
pipeline_stages:
- regex:
expression: '^(?P<timestamp>\d{4}-\d{2}-\d{2}\S\d{2}:\d{2}:\d{2}.\d+\+\d{2}:\d{2}).*'
- timestamp:
source: timestamp
format: RFC3339Nano
action_on_failure: skip