Azure Stream Analytics Job将两个事件中心之间的不匹配记录输出延迟了1分钟

问题描述

谁能帮我,为什么不匹配的记录会延迟1分钟,但是匹配的记录会立即写入博客存储容器中。

即使eventA与其他eventB不匹配,是否有任何方法可以避免延迟(因为我的下游系统将在我的用例中处理)

select b.eventenqueuedutctime as btime,a.Id,a.SysTime,a.UTCTime,b.Id as BId,b.SysTime as BSysTime 
into outputStorage -- to blob storage (container)
from eventA a TIMESTAMP BY eventenqueuedutctime
left outer join eventB b TIMESTAMP BY eventenqueuedutctime
on a.id = b.id
and datediff(minute,b,a) between 0 and 180 -- join with last 3 hours of eventB data

下面是输出,但请查看最后一行(Id:99)currentTime:T19: 42:13.1690000 Z与前4行相比,延迟了1分钟(currentTime:T19: 41:13.1690000 Z)

仅供参考,通过Json序列化通过EventDataBatch一次发送所有eventA ID(2,4,1,101,99)

{"btime":"2020-11-03T17:00:50.6360000Z","Id":2,"SysTime":"2020-11-03T11:41:12.860466-08:00","UTCTime":"2020-11-03T19:41:12.8604646Z","BId":2,"BSysTime":"2020-11-03T09:00:49.6751336-08:00","fullname":"cc","currentTime":"2020-11-03T19:41:13.1690000Z"}
{"btime":"2020-11-03T17:00:50.6360000Z","Id":4,"SysTime":"2020-11-03T11:41:12.8605138-08:00","UTCTime":"2020-11-03T19:41:12.8605135Z","BId":4,"BSysTime":"2020-11-03T09:00:49.6751371-08:00","fullname":null,"Id":1,"SysTime":"2020-11-03T11:41:12.8605561-08:00","UTCTime":"2020-11-03T19:41:12.8605559Z","BId":1,"BSysTime":"2020-11-03T09:00:49.6749841-08:00","fullname":"test","currentTime":"2020-11-03T19:41:13.1690000Z"}
{"btime":"2020-11-03T19:39:04.0100000Z","Id":101,"SysTime":"2020-11-03T11:41:12.860598-08:00","UTCTime":"2020-11-03T19:41:12.8605978Z","BId":101,"BSysTime":"2020-11-03T11:39:03.7462454-08:00","fullname":"test-101","currentTime":"2020-11-03T19:41:13.1690000Z"}
{"btime":null,"Id":99,"SysTime":"2020-11-03T11:41:12.860322-08:00","UTCTime":"2020-11-03T19:41:12.8602803Z","BId":null,"BSysTime":null,"currentTime":"2020-11-03T19:42:13.1690000Z"}

解决方法

这是因为您将JOIN与DATEDIFF一起使用。

使用时间联接,例如DATEDIFF的JOIN:

匹配事件的两面都到达时就会生成匹配。

缺少匹配的数据(例如LEFT OUTER JOIN)会在最后生成 DATEDIFF窗口的左侧,针对左侧的每个事件。

更多详细信息,您可以参考https://docs.microsoft.com/en-us/azure/stream-analytics/stream-analytics-troubleshoot-output#the-first-output-is-delayed