TensorFlowJS 画布大小错误 - 如何显示图像？

问题描述

我正在尝试使用 BodyPix/Tensorflow 来模糊我的网络摄像头的背景。我正在尝试遵循本指南。 https://github.com/vinooniv/video-bg-blur。这显然对他们有用，因为他们有一个活生生的例子。

这是我的代码：

  ngOnInit(): void {
    const videoElement = document.getElementById('face-blur-video') as HTMLVideoElement
    const canvas = document.getElementById('blur-canvas') as HTMLCanvasElement;
    videoElement.onplaying = () => {
      canvas.height = videoElement.height
      canvas.width = videoElement.width
    }
    const context = canvas.getContext('2d');
    context.canvas.height = 320;
    context.canvas.height = 480;
    

    navigator.mediaDevices.getUserMedia({video: true,audio: true})
    .then(stream => {
      videoElement.srcObject = stream;
      videoElement.play();
      const options: ModelConfig = {
        multiplier: .75,outputStride: 16,quantBytes: 4,architecture: 'MobileNetV1'
  
      }
      tfjs.getBackend();
      BodyPix.load(options).then(bp => this.perform(bp))
    })

  async perform(net: BodyPix.BodyPix) {
    while(true) {
      const videoElement = document.getElementById('face-blur-video') as HTMLVideoElement
      const canvas = document.getElementById('blur-canvas') as HTMLCanvasElement;
      const segmentation = await net.segmentPerson(videoElement)
      const backgroundBlurAmount = 6
      const edgeBlurAmount = 2;
      const flipHorizontal = true;
      BodyPix.drawBokehEffect(canvas,videoElement,segmentation,backgroundBlurAmount,edgeBlurAmount,flipHorizontal)
    }

但是，当应用程序运行时，我收到以下错误：

core.js:6162 ERROR Error: Uncaught (in promise): InvalidStateError: Failed to execute 'drawImage' on 'CanvasRenderingContext2D': The image argument is a canvas element with a width or height of 0.
Error: Failed to execute 'drawImage' on 'CanvasRenderingContext2D': The image argument is a canvas element with a width or height of 0.
    at drawWithCompositing (body-pix.esm.js:17)
    at Module.drawBokehEffect (body-pix.esm.js:17)
    at FaceBlurComponent.<anonymous> (face-blur.component.ts:56)
    at Generator.next (<anonymous>)
    at fulfilled (tslib.es6.js:73)
    at ZoneDelegate.invoke (zone-evergreen.js:372)
    at Object.onInvoke (core.js:28510)
    at ZoneDelegate.invoke (zone-evergreen.js:371)
    at Zone.run (zone-evergreen.js:134)
    at zone-evergreen.js:1276
    at resolvePromise (zone-evergreen.js:1213)
    at zone-evergreen.js:1120
    at zone-evergreen.js:1136
    at ZoneDelegate.invoke (zone-evergreen.js:372)
    at Object.onInvoke (core.js:28510)
    at ZoneDelegate.invoke (zone-evergreen.js:371)
    at Zone.run (zone-evergreen.js:134)
    at zone-evergreen.js:1276
    at ZoneDelegate.invokeTask (zone-evergreen.js:406)
    at Object.onInvokeTask (core.js:28497)

我以各种可能的方式设置了宽度和高度。为什么 TensorFlow 检测到画布高度/宽度为零？

解决方法

所以我最终查找了 Google TensorFlow/BodyPix 演示并进行了一些更改。任何更改都不应该有所作为，但是无论修复了什么，一切都可以正常工作。这是我的工作代码：

import { Component,OnInit } from '@angular/core';
import * as BodyPix from '@tensorflow-models/body-pix';
import * as tfjs from '@tensorflow/tfjs';
import { ModelConfig } from '@tensorflow-models/body-pix/dist/body_pix_model';

@Component({
  selector: 'opentok-face-blur',templateUrl: './face-blur.component.html',styleUrls: ['./face-blur.component.scss']
})
export class FaceBlurComponent implements OnInit {

  
  videoElement: HTMLVideoElement;
  canvas : HTMLCanvasElement;
  context 
  constructor() { }

  ngOnInit(): void {

    this.videoElement = document.getElementById('face-blur-video') as HTMLVideoElement
    this.canvas = document.getElementById('blur-canvas') as HTMLCanvasElement;
    this.context = this.canvas.getContext('2d');
    this.context.fillStyle = 'black'
    this.context.fillRect(0,this.canvas.width,this.canvas.height);

    this.bindPage();

  }


  async setupMedia() {
    const stream = await navigator.mediaDevices.getUserMedia({video: true,audio: false});
    this.videoElement.srcObject = stream;

    this.videoElement.onloadedmetadata = () => {
      this.videoElement.width = this.videoElement.videoWidth;
      this.videoElement.height = this.videoElement.videoHeight;
    }
    await this.videoElement.play();
    return this.videoElement;
  }

  async bindPage() {
    tfjs.getBackend();
    const net = await BodyPix.load({
      multiplier: .75,outputStride: 16,quantBytes: 4,architecture: 'MobileNetV1'
    });
    await this.setupMedia();
    this.segmentBodyInRealTime(net);
  }

  async segmentBodyInRealTime(net: BodyPix.BodyPix) {
    const segmentation = await net.segmentPerson(this.videoElement)
    BodyPix.drawBokehEffect(this.canvas,this.videoElement,segmentation,6,2,true);
    this.segmentBodyInRealTime(net);
    
  }
}

我基本上做了一堆异步方法，然后递归调用绘制效果。我不知道为什么这有效，但确实有效。