使用通过make_image_classifier

编程问答 2022-06-05

问题描述

很抱歉，如果这个问题看起来很熟悉，我已经在前面发布了对该问题的更广泛的描述，但是由于我在调查中取得了一些进展并且可以缩小为更具体的问题，因此我将其删除。

上下文：

  • 我正在使用make_image_classifier创建图像分类模型。
  • 我想使用C API加载生成的模型和标签图像。我在这里遇到数据输入问题。
  • 我可以用label_image.py example标记图像，因此模型很好，问题出在我使用C API上。
  • 如果我正确理解make_image_classifier，它将生成一个模型，期望输入4维。我们正在处理图像，所以除了宽度，高度和通道之外，我不知道这第四个维度是什么。缺乏了解可能是我问题的根源。
  • 我在代码中包含一些错误处理，并且在调整大小后尝试从输入缓冲区复制时遇到了我遇到的错误。

问题：

问题1：为什么make_image_classifier生成的模型期望输入4维？有高度，宽度和通道，但是第四个是什么？

当我使用C API执行以下操作以通过图像输入运行模型时：

int inputDims[3] = {224,224,3};
tflStatus = TfLiteInterpreterResizeInputTensor(interpreter,inputDims,3);

我得到：

ERROR: tensorflow/lite/kernels/conv.cc:329 input->dims->size != 4 (3 != 4)
ERROR: Node number 2 (CONV_2D) Failed to prepare.

所以我最终做了：

int inputDims[4] = {1,4);

据我所知，the first dimension size is for the batch size可以处理多个图像。这是正确的吗？

第二季度：我是否应该以调用TfLiteInterpreterResizeInputTensor时所使用的维结构来构造我的数据输入？我在使用此图像RGB输入缓冲区时遇到问题：

// RGB range is 0-255. Scale it to 0-1.
for(int i = 0; i < imageSize; i++){
    imageDataBuffer[i] = (float)pImage[i] / 255.0;
}

在构建类似于给定TfLiteInterpreterResizeInputTensor的输入维的输入时，也会出现错误，但这似乎很愚蠢：

float imageData[1][224][224][3];
int j = 0;
for(int h = 0; h < 224; h++){
  for(int w = 0; w < 224; w++){
    imageData[0][h][w][0] = (float)pImage[j] * (1.0 / 255.0);
    imageData[0][h][w][1] = (float)pImage[j+1] * (1.0 / 255.0);
    imageData[0][h][w][2] = (float)pImage[j+2] * (1.0 / 255.0);

    j = j + 3;
  }
}

最后一个输入结构类似于Python label_image.py在执行此操作时使用的输入结构：

input_data = np.expand_dims(img,axis=0)

问题3：我的输入缓冲区有什么问题，导致TfLiteTensorcopyFromBuffer返回错误代码？

谢谢！

完整代码： 

#include "tensorflow/lite/c/c_api.h"
#include "tensorflow/lite/c/c_api_experimental.h"
#include "tensorflow/lite/c/common.h"
#include "tensorflow/lite/c/builtin_op_data.h"
#include "tensorflow/lite/c/ujpeg.h"

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

// dispose of the model and interpreter objects.
int disposeTfLiteObjects(TfLiteModel* pModel,TfLiteInterpreter* pInterpreter)
{
    if(pModel != NULL)
    {
      TfLiteModelDelete(pModel);
    }

    if(pInterpreter)
    {
      TfLiteInterpreterDelete(pInterpreter);
    }
}

// The main function.
int main(void) 
{
    TfLiteStatus tflStatus;

    // Create JPEG image object.
    ujImage img = ujCreate();

    // Decode the JPEG file.
    ujDecodeFile(img,"image_224x224.jpeg");

    // Check if decoding was successful.
    if(ujIsValid(img) == 0){
        return 1;
    }
    
    // There will always be 3 channels.
    int channel = 3;

    // Height will always be 224,no need for resizing.
    int height = ujGetHeight(img);

    // Width will always be 224,no need for resizing.
    int width = ujGetWidth(img);

    // The image size is channel * height * width.
    int imageSize = ujGetimageSize(img);

    // Fetch RGB data from the decoded JPEG image input file.
    uint8_t* pImage = (uint8_t*)ujGetimage(img,NULL);

    // The array that will collect the JPEG RGB values.
    float imageDataBuffer[imageSize];

    // RGB range is 0-255. Scale it to 0-1.
    int j=0;
    for(int i = 0; i < imageSize; i++){
        imageDataBuffer[i] = (float)pImage[i] / 255.0;
    }

    // Load model.
    TfLiteModel* model = TfLiteModelCreateFromFile("model.tflite");

    // Create the interpreter.
    TfLiteInterpreter* interpreter = TfLiteInterpreterCreate(model,NULL);

    // Allocate tensors.
    tflStatus = TfLiteInterpreterallocateTensors(interpreter);

    // Log and exit in case of error.
    if(tflStatus != kTfLiteOk)
    {
      printf("Error allocating tensors.\n");
      disposeTfLiteObjects(model,interpreter);
      return 1;
    }
    
    int inputDims[4] = {1,3};
    tflStatus = TfLiteInterpreterResizeInputTensor(interpreter,4);

    // Log and exit in case of error.
    if(tflStatus != kTfLiteOk)
    {
      printf("Error resizing tensor.\n");
      disposeTfLiteObjects(model,interpreter);
      return 1;
    }

    tflStatus = TfLiteInterpreterallocateTensors(interpreter);

    // Log and exit in case of error.
    if(tflStatus != kTfLiteOk)
    {
      printf("Error allocating tensors after resize.\n");
      disposeTfLiteObjects(model,interpreter);
      return 1;
    }

    // The input tensor.
    TfLiteTensor* inputTensor = TfLiteInterpreterGetInputTensor(interpreter,0);

    // copy the JPEG image data into into the input tensor.
    tflStatus = TfLiteTensorcopyFromBuffer(inputTensor,imageDataBuffer,imageSize);
    
    // Log and exit in case of error.
    // FIXME: Error occurs here.
    if(tflStatus != kTfLiteOk)
    {
      printf("Error copying input from buffer.\n");
      disposeTfLiteObjects(model,interpreter);
      return 1;
    }

    // Invoke interpreter.
    tflStatus = TfLiteInterpreterInvoke(interpreter);

    // Log and exit in case of error.
    if(tflStatus != kTfLiteOk)
    {
      printf("Error invoking interpreter.\n");
      disposeTfLiteObjects(model,interpreter);
      return 1;
    }

    // Extract the output tensor data.
    const TfLiteTensor* outputTensor = TfLiteInterpreterGetoutputTensor(interpreter,0);

    // There are three possible labels. Size the output accordingly.
    float output[3];

    tflStatus = TfLiteTensorcopyToBuffer(outputTensor,output,3 * sizeof(float));

    // Log and exit in case of error.
    if(tflStatus != kTfLiteOk)
    {
      printf("Error copying output to buffer.\n");
      disposeTfLiteObjects(model,interpreter);
      return 1;
    }

    // Print out classification result.
    printf("Confidences: %f,%f,%f.\n",output[0],output[1],output[2]); 

    // dispose of the TensorFlow objects.
    disposeTfLiteObjects(model,interpreter);
    
    // dispoice of the image object.
    ujFree(img);
    
    return 0;
}

编辑＃1：好吧，在TfLiteTensorcopyFromBuffer里面：

TfLiteStatus TfLiteTensorcopyFromBuffer(TfLiteTensor* tensor,const void* input_data,size_t input_data_size) {
    if (tensor->bytes != input_data_size) {
        return kTfLiteError;
    }

    memcpy(tensor->data.raw,input_data,input_data_size);
    return kTfLiteOk;
}

我的input_data_size值为150,528（3通道x 224像素高度x 224像素宽度），但是tensor->bytes是602,112（3通道x 448像素高度x 224像素448，我假设是？）。我不理解这种差异，特别是因为我用TfLiteInterpreterResizeInputTensor调用了{1,3}。

编辑＃2：我相信我已经找到了答案here。确认后将解决此帖子。

解决方法

我在EDIT＃2上链接到的解决方案就是答案。最后，我只需要替换：

{ "abi-blacklist": [ "stdcall","fastcall","vectorcall","thiscall","win64","sysv64" ],"arch": "arm","data-layout": "e-m:e-p:32:32-Fi8-i64:64-v128:64:128-a:0:32-n32-S64","dynamic-linking": true,"env": "uclibc","executables": true,"features": "+v7,+vfp3,-d32,+thumb2,-neon","has-elf-tls": true,"has-rpath": true,"linker-flavor": "gcc","linker-is-gnu": true,"llvm-target": "armv7-unknown-linux-gnueabihf","max-atomic-width": 64,"os": "linux","position-independent-executables": true,"pre-link-args": { "gcc": [ "-Wl,--as-needed","-Wl,-z,noexecstack" ] },"relro-level": "full","target-c-int-width": "32","target-endian": "little","target-family": "unix","target-mcount": "\u0001__gnu_mcount_nc","target-pointer-width": "32","vendor": "unknown" }

具有：

var marker; mymap.on('click',function(e)=>{ if(marker){ marker.setLatLng(e.latlng); }else{ marker = L.marker(e.latlng).addTo(mymap); } })

干杯！

c-apiimage-classificationmachine-learningtensorflowtensorflow-lite