以微秒精度插入数据帧

问题描述

我正在尝试将传感器数据写入 InfluxDB。数据需要纳秒精度。导入的日期不正确,未绘制所有数据点。

重现步骤:

我将 json file 读入数据帧,然后将其发送到 influxdb。要重现的代码

 dataDF = pd.read_json(dataFile)
  startTime = np.datetime64(dataDF['TimeStamp'][0])
  period = round(1e9 / dataDF['SampleRate'][0])
  timeDF = [startTime]

  for i in range(dataDF.shape[0]):
      ts = timeDF[i] + np.timedelta64(period,'ns')
      timeDF.append(ts)

  dataDF.index = timeDF[:-1]
  dataDF = dataDF.drop('TimeStamp',1)
  
  write_api.write(bucket="devBucket",record=dataDF,data_frame_measurement_name=dataDF['DeviceType'][0],data_frame_tag_columns=['SerialNumber','SampleRate','DeviceType'])

我不确定我是否使用了错误的库,但将其与示例进行比较,它似乎应该可以工作。 这是插入前的数据帧片段。

                           SerialNumber   DeviceType  SampleRate   Data
2021-02-12 13:25:35.000000000    1234567890   Pressure       44000  1073741816
2021-02-12 13:25:35.000022727    1234567890   Pressure       44000  1073791012
2021-02-12 13:25:35.000045454    1234567890   Pressure       44000  1073791012
2021-02-12 13:25:35.000068181    1234567890   Pressure       44000  1073745384
2021-02-12 13:25:35.000090908    1234567890   Pressure       44000  1073745384

解决方法

我只是尝试按照 Python 脚本导入您的数据,一切看起来都不错

npx react-native init AwesomeTSProject --template react-native-template-typescript

您的所有数据在时间范围内都有时间戳:2021-02-12T12:42:09Z - 2021-02-12T12:42:10Z。所以你应该使用脚本编辑器来创建正确的查询:

import numpy as np
import pandas as pd

from influxdb_client import InfluxDBClient
from influxdb_client.client.write_api import SYNCHRONOUS

"""
Load DataFrame form CSV File
"""
dataDF = pd.read_json("sample.json")
startTime = np.datetime64(dataDF['TimeStamp'][0])
period = round(1e9 / dataDF['SampleRate'][0])
timeDF = [startTime]

for i in range(dataDF.shape[0]):
    ts = timeDF[i] + np.timedelta64(period,'ns')
    timeDF.append(ts)

dataDF.index = timeDF[:-1]
dataDF = dataDF.drop('TimeStamp',1)

print(dataDF)

client = InfluxDBClient(url="http://localhost:8086",token="my-token",org="my-org")

write_api = client.write_api(write_options=SYNCHRONOUS)
write_api.write(bucket="my-bucket",record=dataDF,data_frame_measurement_name=dataDF['DeviceType'][0],data_frame_tag_columns=['SerialNumber','SampleRate','DeviceType'])

"""
Querying ingested data
"""
query = 'from(bucket: "my-bucket")' \
        ' |> range(start: 0,stop: now())' \
        ' |> filter(fn: (r) => r._measurement == "Pressure")' \
        ' |> pivot(rowKey:["_time"],columnKey: ["_field"],valueColumn: "_value")' \
        ' |> count(column: "Data")'
tables = client.query_api().query(query=query)

"""
Processing results
"""
print()
print("=== count of ingested items: ===")
print(tables[0].records[0]["Data"])

"""
Close client
"""
client.__del__()