问题描述
我有一个由事件网格事件触发的Azure函数。仅当将Blob上传到存储帐户时,才会创建事件网格事件。
此功能现在已部署并运行良好,尽管由于某种原因,即使该功能已成功处理,该功能也会不断被同一事件触发?
示例:
代码段:
def main(event: func.EventGridEvent):
result = json.dumps({
'id' : event.id,'data' : event.get_json(),'topic' : event.topic,'subject' : event.subject,'event_type' : event.event_type
})
logging.info('EventGrid trigger processing an event: %s',result)
credential = DefaultAzureCredential()
download_start_time = datetime.datetime.now()
logging.info(f'######## Starting downloading blob from storage at ' + str(download_start_time) + ' ########')
# =============================
# Download blob from storage container:
# =============================
blob_client = BlobClient.from_blob_url(event.get_json()["url"],credential)
blob_data = blob_client.download_blob().readall()
blob_byte_stream = io.BytesIO(blob_data)
编辑1:这仍在发生,这次有所不同。
- 现在,成功传递消息并运行函数后,EventGrid会继续触发
我该如何调试?
解决方法
我终于弄清楚问题出在哪里……在使用Azure Storage Explorer测试Blob上传时,使用BlobClient.from_blob_url
方法可以很好地工作。但是,当使用Azure Data Factory时,将使用其他API,并且EventGrid消息中的data.url
属性不是实际的Blob URL(包含dfs
而不是blob
)。
奇怪的是,在我将此问题提交给支持团队后不久,一个新的blobUrl
属性被添加到EventGrid data
对象中。
在我的代码中,我只是将“ url”更改为“ blobUrl”,该方法成功了。 (我还改进了Python代码的错误处理,以适应将来出现的此类错误。)
Documented EventGrid消息(截至2020年12月10日):
- 没有
blobUrl
属性
[{
"topic": "/subscriptions/{subscription-id}/resourceGroups/Storage/providers/Microsoft.Storage/storageAccounts/my-storage-account","subject": "/blobServices/default/containers/my-file-system/blobs/new-file.txt","eventType": "Microsoft.Storage.BlobCreated","eventTime": "2017-06-26T18:41:00.9584103Z","id": "831e1650-001e-001b-66ab-eeb76e069631","data": {
"api": "CreateFile","clientRequestId": "6d79dbfb-0e37-4fc4-981f-442c9ca65760","requestId": "831e1650-001e-001b-66ab-eeb76e000000","eTag": "\"0x8D4BCC2E4835CD0\"","contentType": "text/plain","contentLength": 0,"contentOffset": 0,"blobType": "BlockBlob","url": "https://my-storage-account.dfs.core.windows.net/my-file-system/new-file.txt","sequencer": "00000000000004420000000000028963","storageDiagnostics": {
"batchId": "b68529f3-68cd-4744-baa4-3c0498ec19f0"
}
},"dataVersion": "2","metadataVersion": "1"
}]
实际出现的EventGrid消息:
- 通知
blobUrl
已添加到架构中
{
"id": "long-string","data": {
"api": "CreateFile","requestId": "long-string","eTag": "0x8D89B4FF7150079","contentType": "application/octet-stream","blobProperties": [{
"acl": [{
"access": "u::rw,u:long-string:rwx,g::rx,g:long-string:rx,m::rw,o::","permission": "0660","owner": "long-string","group": "$superuser"
}]
}],"blobUrl": "https://myfunction.blob.core.windows.net/container/20201208/730420201208080239.csv","url": "https://myfunction.dfs.core.windows.net/container/20201208/730420201208080239.csv","sequencer": "0000000000000000000000000000692c00000000000e1a99","identity": "long-string","storageDiagnostics": {
"batchId": "long-string"
}
},"topic": "/subscriptions/long-string/resourceGroups/myResourceGroup/providers/Microsoft.Storage/storageAccounts/storageAccount","subject": "/blobServices/default/containers/container/blobs/20201208/730420201208080239.csv","event_type": "Microsoft.Storage.BlobCreated"
}
此处的另一项警告...请注意以上消息中的Content-Length
-
CreateFile
API不是实际消息,它表明已创建了Blob。 -
FlushWithClose
API是
文档中对此有一个注释。因此,我还必须设置一个EventGrid高级过滤器,该过滤器仅在生成FlushWithClose
个事件时才触发(!)