TPL Dataflow 流水线中的图像刷新问题

编程问答 2022-12-04

问题描述

我使用 TPL Dataflow 从路径加载视频（我使用 Emgu.CV 库加载），并通过 TPL Dataflow 首先在 Windows 窗体应用程序中绘制它（之后将有一个与木板）。我的另一篇文章对 TPL 数据流帮助很大： Asynchronous Task,video buffering

但是在设置 TPL 数据流后，第一个图像仅加载到 GUI 中，之后（虽然 Block 似乎正在运行，因为打印显示在 cmd 中）图像没有刷新......我不明白出了什么问题?它与调度程序或 TPL 数据流有关吗？ 代码如下：

public async void CreateVideoProcessingNetwork()
{
    string video_path = @"C:\.......\video_640x360_360p.mp4";

    /* displayVideo Block*/
    var display_video = new ActionBlock<Bitmap>(async received_bitmap =>
    {
        Console.WriteLine("Inside display_video");

        PicturePlot2.Refresh();
        PicturePlot2.Image = received_bitmap;
        Console.WriteLine("Image size = " + received_bitmap.Size);
        Console.WriteLine("Image width = " + received_bitmap.Width);

        await Task.Delay(30);
   });

    var loadVideo = new ActionBlock<string>(async path =>
    {
        capture = new VideoCapture(path);
        Mat matrix = new Mat();
        capture.Read(matrix);
        var mem_stream = new MemoryStream();
        Bitmap HWimage;

        while (matrix.Rows != 0 && matrix.Width != 0)
        {
            Console.WriteLine("Inside LoadVideo");
            matrix = new Mat();
            capture.Read(matrix);
            Bitmap bitmap = new Bitmap(matrix.Width,matrix.Rows);
            bitmap = matrix.ToBitmap();
            bitmap.Save(mem_stream,System.Drawing.Imaging.ImageFormat.Jpeg);
            byte[] image_array = mem_stream.ToArray();
            Console.WriteLine("image_array = " + image_array.Length);
            using (var mem_stream_hw = new MemoryStream(image_array)) HWimage = new Bitmap(mem_stream_hw);
            var accepted = await display_video.SendAsync(HWimage);
            if (!accepted) break;
            await Task.Delay(25);
        }
    });

    PropagateCompletion(loadVideo,display_video);

    loadVideo.Post(video_path);
    loadVideo.Complete();
    await display_video.Completion;
}

我理解错了吗？我想通过 TPL 数据流在视频显示中进行某种流水线操作。

更新：

在我做了提到的改变之后，视频会播放。但还有一个问题。 “生产者”的数据创建速度更快（实际上它取决于 TransformManyBlock 中的 Thread.Sleep 时间）并且在播放视频几秒钟后应用程序崩溃并出现以下错误：

Unhandled Exception: System.Runtime.InteropServices.ExternalException: A generic error occurred in GDI+.
   at system.drawing.graphics.MeasureString(String text,Font font,Sizef layoutArea,StringFormat stringFormat)
   at system.drawing.graphics.MeasureString(String text,Int32 width)
   at System.Windows.Forms.ThreadExceptionDialog..ctor(Exception t)
   at System.Windows.Forms.Application.threadcontext.OnThreadException(Exception t)
   at System.Windows.Forms.Control.WndProcException(Exception e)
   at System.Windows.Forms.Control.ControlNativeWindow.OnThreadException(Exception e)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd,Int32 msg,IntPtr wparam,IntPtr lparam)
   at System.Windows.Forms.UnsafeNativeMethods.dispatchMessageW(MSG& msg)
   at System.Windows.Forms.Application.ComponentManager.System.Windows.Forms.UnsafeNativeMethods.imsoComponentManager.FPushMessageLoop(IntPtr dwComponentID,Int32 reason,Int32 pvLoopData)
   at System.Windows.Forms.Application.threadcontext.RunMessageLoopInner(Int32 reason,ApplicationContext context)
   at System.Windows.Forms.Application.threadcontext.RunMessageLoop(Int32 reason,ApplicationContext context)
   at System.Windows.Forms.Application.Run(Form mainForm)
   at ntComlabGUI.Program.Main()

例如，如果 Thread.Sleep 被删除，错误几乎立即发生（在前 1-2 秒内）。有人怎么做流量控制？在 Asynchronous Task,video buffering 帖子中提到了 BoundedCapacity 方法。但我试过了，错误仍然存​​在。代码如下：

public async void CreateVideoProcessingNetwork()
{
    //string video_path = @"C:\Projects_Repo\ComlabDMA_Ethernet_video\ntComlabGUI_UltRascale_ethernet\ntComlabGUI\video_640x360_360p.mp4";
    string video_path = @"C:\Projects_Repo\ComlabDMA_Ethernet_video\ntComlabGUI_UltRascale_ethernet\ntComlabGUI\video_640x360_360p.mp4";


    /* Video Loading TPL Block */
    var video_loader = new TransformManyBlock<string,Bitmap>(load_video,new ExecutionDataflowBlockOptions { BoundedCapacity = 128 });

    IEnumerable<Bitmap> load_video(string path)
    {

        capture = new VideoCapture(path);
        Mat matrix = new Mat();
        capture.Read(matrix);
        var mem_stream = new MemoryStream();


        while (matrix.Rows != 0 && matrix.Width != 0)
        {
            capture.Read(matrix);
            Bitmap bitmap = new Bitmap(matrix.Width,matrix.Rows);

            bitmap = matrix.ToBitmap();

            yield return bitmap;
            //Thread.Sleep(1);

        }
        yield break;
    }



    /* Video Loading TPL Block */

    var display_video = new ActionBlock<Bitmap>(async received_image =>
    {
        PicturePlot2.Image = received_image;
        await Task.Delay(33);
    },new ExecutionDataflowBlockOptions()
    {
        TaskScheduler = TaskScheduler.FromCurrentSynchronizationContext(),BoundedCapacity = 128
    });



    var linkOptions = new DataflowLinkOptions { PropagateCompletion = true };

    video_loader.LinkTo(display_video,linkOptions);
    video_loader.Post(video_path);
    video_loader.Complete();
    await display_video.Completion;


}

也许以下链接是解决方案？ ：How do I arrange flow control in TPL Dataflows?

预先感谢您的帮助， 并且为了快速响应， 真的很感谢！

解决方法

最有可能的问题是 PicturePlot2 组件不喜欢被非 UI 线程操作。为了确保在 UI 线程上调用 ActionBlock 的委托，您可以像这样配置块的 TaskScheduler 选项：

var display_video = new ActionBlock<Bitmap>(async received_bitmap =>
{
    Console.WriteLine("Inside display_video");

    PicturePlot2.Refresh();
    PicturePlot2.Image = received_bitmap;
    Console.WriteLine("Image size = " + received_bitmap.Size);
    Console.WriteLine("Image width = " + received_bitmap.Width);

    await Task.Delay(30);
},new ExecutionDataflowBlockOptions()
{
    TaskScheduler = TaskScheduler.FromCurrentSynchronizationContext()
});

为此，需要在 UI 线程上实例化该块。这是因为 Windows 窗体应用程序在启动时会在 UI 线程上安装 special SynchronizationContext，我们希望 TaskScheduler.FromCurrentSynchronizationContext 方法捕获此上下文。

c#c#task-parallel-librarytpl-dataflow