问题描述
我想创建一个浏览器扩展,允许用户在任何使用javascript网络api的网站上向其视频/音频流添加效果,而无需特殊的插件。
Google搜索并不是特别有用,所以我开始怀疑这是否有可能。
我在这里有2个主要问题:
-
javascript + chrome可以吗?
-
任何指向其他资源的链接都将受到赞赏。
解决方法
我不是很喜欢网络扩展,因此甚至可能有一个更简单的API,并且我不会详细介绍实现,但从理论上讲您确实可以做到。
所要做的就是覆盖获取MediaStream的方法,将原始MediaStream绘制到可以应用过滤器的HTML画布上,然后简单地返回一个由以下组成的新MediaStream画布元素的captureStream()
中的MediaStream的VideoTrack,以及原始MediaStream中的其他轨道。
用于gUM的非常基本的概念实现证明如下:
// overrides getUserMedia so it applies an invert filter on the videoTrack
{
const mediaDevices = navigator.mediaDevices;
const original_gUM = mediaDevices.getUserMedia.bind(mediaDevices);
mediaDevices.getUserMedia = async (...args) => {
const original_stream = await original_gUM(...args);
// no video track,no filter
if( !original_stream.getVideoTracks().length ) {
return original_stream;
}
// prepare our DOM elements
const canvas = document.createElement('canvas');
const ctx = canvas.getContext('2d');
const video = document.createElement('video');
// a flag to know if we should keep drawing on the canvas or not
let should_draw = true;
// no need for audio there
video.muted = true;
// gUM video tracks can change size
video.onresize = (evt) => {
canvas.width = video.videoWidth;
canvas.height = video.videoHeight;
};
// in case users blocks the camera?
video.onpause = (evt) => {
should_draw = false;
};
video.onplaying = (evt) => {
should_draw = true;
drawVideoToCanvas();
};
video.srcObject = original_stream;
await video.play();
const canvas_track = canvas.captureStream().getVideoTracks()[0];
const originalStop = canvas_track.stop.bind(canvas_track);
// override the #stop method so we can revoke the camera stream
canvas_track.stop = () => {
originalStop();
should_draw = false;
original_stream.getVideoTracks()[0].stop();
};
// merge with audio tracks
return new MediaStream( original_stream.getAudioTracks().concat( canvas_track ) );
// the drawing loop
function drawVideoToCanvas() {
if(!should_draw) {
return;
}
ctx.filter = "none";
ctx.clearRect(0,canvas.width,canvas.height);
ctx.filter = "invert(100%)";
ctx.drawImage(video,0);
requestAnimationFrame( drawVideoToCanvas );
}
};
}
然后,每个调用此方法的脚本都将收到一个经过过滤的videoTrack。
Outsourced example since gUM is not friend with StackSnippets.
现在,我不确定如何从Web扩展中覆盖方法,您必须自己学习,并且当心此脚本实际上只是概念证明,还没有准备好投入生产。除了演示案例,我什么也没做。