问题描述
我正在尝试让 GStreamer + OpenCV RTSP 视频捕获在基于 NVIDIA PyTorch 图像的 Docker 容器中工作。我不得不从源代码构建 OpenCV 以启用 GStreamer 集成,我在我的 Dockerfile 中这样做:
FROM nvcr.io/nvidia/pytorch:19.12-py3
# OpenCV custom build instructions from:
# https://medium.com/@galaktyk01/how-to-build-opencv-with-gstreamer-b11668fa09c
# https://github.com/junjuew/Docker-OpenCV-GStreamer/blob/master/opencv3-gstreamer1.0-Dockerfile
# Install base dependencies + gstreamer
RUN pip uninstall -y opencv-python
RUN apt-get update
RUN apt-get -y install build-essential
RUN apt-get -y install pkg-config
RUN apt-get install -y libgstreamer1.0-0 \
gstreamer1.0-plugins-base \
gstreamer1.0-plugins-good \
gstreamer1.0-plugins-bad \
gstreamer1.0-plugins-ugly \
gstreamer1.0-libav \
gstreamer1.0-doc \
gstreamer1.0-tools \
libgstreamer1.0-dev \
libgstreamer-plugins-base1.0-dev \
cmake \
protobuf-compiler \
libgtk2.0-dev \
ocl-icd-opencl-dev
# Clone OpenCV repo
workdir /
RUN git clone https://github.com/opencv/opencv.git
workdir /opencv
RUN git checkout 4.2.0
# Build OpenCV
RUN mkdir /opencv/build
workdir /opencv/build
RUN ln -s /opt/conda/lib/python3.6/site-packages/numpy/core/include/numpy /usr/include/numpy
RUN cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D INSTALL_C_EXAMPLES=OFF \
-D PYTHON_EXECUTABLE=$(which python) \
-D BUILD_opencv_python2=OFF \
-D CMAKE_INSTALL_PREFIX=$(python -c "import sys; print(sys.prefix)") \
-D python3_EXECUTABLE=$(which python3) \
-D python3_INCLUDE_DIR=$(python -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") \
-D python3_PACKAGES_PATH=$(python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") \
-D WITH_GSTREAMER=ON \
-D BUILD_EXAMPLES=ON ..
RUN make -j$(nproc)
# Install OpenCV
RUN make install
RUN ldconfig
这构建成功,如果我从 Docker 容器中打印 OpenCV 的构建信息,GStreamer 显示为可用:
python -c 'import cv2; print(cv2.getBuildinformation());'
/* snip */
Video I/O:
DC1394: NO
FFMPEG: NO
avcodec: NO
avformat: NO
avutil: NO
swscale: NO
avresample: NO
GStreamer: YES (1.14.5)
v4l/v4l2: YES (linux/videodev2.h)
但是,一旦我尝试在 Docker 容器中使用带有 cv2.VideoCapture()
的 GStreamer 管道,它立即失败:
import cv2
video = cv2.VideoCapture('gst-launch-1.0 rtspsrc location=<<rtsp URL>> latency=0 ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink',cv2.CAP_GSTREAMER)
我收到此“警告”(例如,错误)。我无法从 RTSP 提要中提取帧。
[ WARN:0] global /opencv/modules/videoio/src/cap_gstreamer.cpp (713) open OpenCV | GStreamer warning: Error opening bin: unexpected reference "gst-launch-1" - ignoring
[ WARN:0] global /opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
如果我在 Docker 容器之外执行此操作,它就像一个魅力。另请注意,如果我从 Docker 容器内的命令行运行 GStreamer 管道,我会得到与在 Docker 容器外运行相同命令相同的合理输出:
root:/# gst-launch-1.0 rtspsrc location=<<rtsp URL>> latency=0 ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink
Setting pipeline to PAUSED ...
Pipeline is live and does not need preroll ...
Progress: (open) opening Stream
Progress: (connect) Connecting to <<rtsp URL>>
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Redistribute latency...
Redistribute latency...
在调试 GStreamer 无法与 OpenCV 的 VideoCapture
一起使用的问题方面,我不确定下一步该怎么做 - 有什么建议吗?
解决方法
我认为您不应该在 gst-launch-1.0
管道描述中包含 cv2
命令行工具。
它只需要唯一的 GStreamer 管道,而不是控制台命令。例如:
gst_str = ('v4l2src device=/dev/video{} ! '
'video/x-raw,width=(int){},height=(int){} ! '
'videoconvert ! appsink').format(dev,width,height)
return cv2.VideoCapture(gst_str,cv2.CAP_GSTREAMER)
所以在你的情况下尝试:
import cv2
video = cv2.VideoCapture('rtspsrc location=<<rtsp URL>> latency=0 ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink',cv2.CAP_GSTREAMER)
,
我有一些建议。但请注意,我对 Gstreamer 没有太多经验。
- 这可能与提要的分辨率有关吗?来自here:
这与 docker 无关。我将我的 Android 手机用作网络摄像头(使用 adb-ffmpeg-v4l2loopback)并且我刚刚使用了错误的 640x360 分辨率而不是 1280x720(从我的 bash 历史记录中选择了错误的行)。
-
您是否尝试过安装 opencv_contrib?它可能无关紧要,但我看到通过安装它解决了很多 opencv 问题。
-
也许可以尝试添加参数
format=(string)NV12
,来源:OpenCV VideoCapture not working with GStreamer plugin -
这是来自 Python cv2.CAP_GSTREAMER Examples 的第一个例子:
def open_cam_rtsp(uri,height,latency):
"""Open an RTSP URI (IP CAM)."""
gst_elements = str(subprocess.check_output('gst-inspect-1.0'))
if 'omxh264dec' in gst_elements:
# Use hardware H.264 decoder on Jetson platforms
gst_str = ('rtspsrc location={} latency={} ! '
'rtph264depay ! h264parse ! omxh264dec ! '
'nvvidconv ! '
'video/x-raw,height=(int){},'
'format=(string)BGRx ! videoconvert ! '
'appsink').format(uri,latency,height)
elif 'avdec_h264' in gst_elements:
# Otherwise try to use the software decoder 'avdec_h264'
# NOTE: in case resizing images is necessary,try adding
# a 'videoscale' into the pipeline
gst_str = ('rtspsrc location={} latency={} ! '
'rtph264depay ! h264parse ! avdec_h264 ! '
'videoconvert ! appsink').format(uri,latency)
else:
raise RuntimeError('H.264 decoder not found!')
return cv2.VideoCapture(gst_str,cv2.CAP_GSTREAMER)
他们似乎检查 subprocess.check_output('gst-inspect-1.0')
的输出以获取 Gstreamer 元素,并相应地设置 gst_str
。
- 在上面链接中的 13 个示例中,我没有看到
gst-launch-1.0
命令。是否会导致问题?
这个链接有帮助吗? Docker Error Could not capture frame on Ubuntu 18.04
抱歉,这些都没有帮助。希望你早日找到解决办法!