MediaPipe 如何使用BoxTracking aar?

问题描述

刚接触 MediaPipe,对图、节点、子图等概念不熟悉

构建BoxTracking的aar文件后,由于模型需要一些未知的输入和输出参数,无法在基于Android Studio gradle的项目中运行它

关于比较HandTracking graphBoxTracking graph 使用 visualizer 工具,并使用 a working project of HandTracking 将 aar 文件添加为 lib,我添加了新的所需输入流和侧包,如图所示。

结果总是一些错误,主要是由于输入错误BoxTracking 是直接使用的子图。如何知道需要哪个输入和输入的数据类型来运行它?

2021-02-05 21:15:23.477 22514-22564/com.example.mediapipemultihandstrackingapp E/FrameProcessor: Mediapipe error: 
com.google.mediapipe.framework.MediaPipeException: internal: Graph has errors: 
Calculator::open() for node "objectdetectionsubgraphgpu__TfLiteInferenceCalculator" Failed: ; Could not read asset: ssdlite_object_detection.tfliteer_util.cc:158) 
    at com.google.mediapipe.framework.Graph.nativeMovePacketToInputStream(Native Method)
    at com.google.mediapipe.framework.Graph.addConsumablePacketToInputStream(Graph.java:360)
    at com.google.mediapipe.components.FrameProcessor.onNewFrame(FrameProcessor.java:442)
    at com.google.mediapipe.components.ExternalTextureConverter$renderThread.renderNext(ExternalTextureConverter.java:364)
    at com.google.mediapipe.components.ExternalTextureConverter$renderThread.lambda$onFrameAvailable$0$ExternalTextureConverter$renderThread(ExternalTextureConverter.java:309)
    at com.google.mediapipe.components.-$$Lambda$ExternalTextureConverter$renderThread$Y1vV_XyLsWZ0ebOvq-iwjQ0H3Sw.run(UnkNown Source:4)
    at android.os.Handler.handleCallback(Handler.java:883)
    at android.os.Handler.dispatchMessage(Handler.java:100)
    at android.os.Looper.loop(Looper.java:237)
    at com.google.mediapipe.glutil.GlThread.run(GlThread.java:141)

解决方法

作为输入和输出所需的数据类型未包含在默认构建中,必须修改构建配置以包含 box_tracker.proto 及其依赖项。

https://github.com/google/mediapipe/issues/1624