Tensorflow.js裁剪图像返回零张量

问题描述

我有一个自定义模型，该模型吸收了Blazeface模型中的裁剪脸部，然后输出3个类别的预测。

在将其发送到我的自定义模型之前，我将裁剪后的面孔调整为[1,224,3]形状

每次预测时输出：

Float32Array [
  6.522771905936864e-11,3.698188456857654e-12,1,]

调整裁切后的面孔大小并进行预测的代码：

 const getPrediction = async tensor => {
    if (!tensor) {
      console.log("Tensor not found!");
    }
    // Load both models 
    const bfModel = await blazefaceModel;
    const returnTensors = true;
    const faces = await bfModel
      .estimateFaces(tensor,returnTensors)
      .catch(e => console.log(e));
    // Reshape tensor from rank 3 to rank 4
    const tensorReshaped = tensor.reshape([1,3]);
    const scale = {
      height: styles.camera.height / tensorDims.height,width: styles.camera.width / tensorDims.width
    };

    // Faces is an array of objects
    if (!isEmpty(faces)) {
      // Setting faces in state
      setModelFaces({ faces });
    } 
    //Looping over array of objects in faces
    faces.map((face,i) => {
      const { topLeft,bottomright } = face;
      const width = Math.floor(
        bottomright.dataSync()[0] - topLeft.dataSync()[0] * scale.width
      );
      const height = Math.floor(
        bottomright.dataSync()[1] - topLeft.dataSync()[1] * scale.height
      );
      const Boxes = tf
        .concat([topLeft.dataSync(),bottomright.dataSync()])
        .reshape([-1,4]); 
      // Cropping out faces from original tensor 
      const crop = tf.image.cropAndResize(
        tensorReshaped,Boxes,[0],[height,width]
      );
      // Resize cropped faces to [1,3]
      const alignCorners = true;
      const imageResize = tf.image.resizeBilinear(
        crop,[224,224],alignCorners
      );
      makePrediction(imageResize);
    });
  };
// Make predictions on the tensors
  const makePrediction = async image => {
    if (!image) {
      console.log("No input!");
    }
    const model = await loadedModel;
    const prediction = await model.predict(image,{ batchSize: 1 });
    if (!prediction || isEmpty(prediction)) {
      console.log("Prediction not available");
    }
    console.log(prediction);
    console.log(prediction.dataSync());
  };

编辑

我在将预测设为1时仍然尝试更改批次大小，并且仍然是相同的问题

我尝试将keras模型转换为tfjs格式，并且仍然存在相同问题

我在做出预测后尝试处理张量，但仍然存在错误

所以我打印出调整后的面孔的张量及其很多0'

Tensor before prediction
Tensor
    [[[[0,0],[0,...,0]],[[0,...
      [[0,0]]]]
undefined
Tensor during prediction
Tensor
    [[[[0,0]]]]
undefined

解决方法

boxes中的

tf.image.cropAndResize是介于0和1之间的归一化坐标。因此应该使用[imageWidth,imageHeight]来归整topLeft和bottomRight

normalizedTopLeft = topLeft.div(tensor.shape.slice(-3,-2)) 
// slice will get [h,w] of a tensor of shape [b,h,w,ch] or [h,ch]

// do likewise for bottomRight
// use normalizedTopLeft instead of topLeft for cropping