如何使用PipeReader读取JSON平面文件?

问题描述

我正在尝试学习这个新的system.io.pipelines,以及用于反序列化json的新的webapi策略...

我编写了自己的JsonConverter,但是我不知道从json平面文件固定装置初始化Utf9JsonReader的正确方法

这是测试:

    [Fact]
    public void WhenGivenjsonObjectThenEntityDTOReturned() {
                    
        using(var stream = new FileStream("Fixtures/BookStoreJson.json",FileMode.Open))
        {
            var pipe = PipeReader.Create(stream);
            ReadResult bytes;
            pipe.TryRead(out bytes);
            var reader = new Utf8JsonReader(bytes.Buffer);
            var target = new EntityDTOConverter();
            reader.Read();
            var actual = target.Read(ref reader,typeof(EntityDTO),new JsonSerializerOptions());
            
            Assert.True(actual.Props.ContainsKey("name"));
        }
    

    }

当我对此进行调试时,即使BookStoreJson.json文件包含以下内容,bytes.buffer也被设置为0字节:

{
    "name": "Tattered Cover","store":{
       "book":[
          {
             "category":"reference","author":"Nigel Rees","title":"Sayings of the Century","price":8.95
          },{
             "category":"fiction","author":"Evelyn Waugh","title":"Sword of Honour","price":12.99
          },"author":"J. R. R. Tolkien","title":"The Lord of the Rings","isbn":"0-395-19395-8","price":22.99
          }
       ],"bicycle":{
          "color":"red","price":19.95
       }
    }
 }

解决方法

抱歉,我没有意识到异步过程,我正在控制台应用程序中进行测试。除了不确定该答案对您有多大帮助外,基本上您可以通过访问结果来同步运行异步任务。缓冲区大小也有限制,如果json文件大小较大,则可能需要创建自定义池,并使用AdvanceTo选项来读取缓冲区的末尾以获取流。

using System;
using System.Buffers;
using System.IO;
using System.IO.Pipelines;
using System.Text;
using System.Text.Json;
using System.Threading.Tasks;
using Xunit;
 

            public class Person
            {
                public string Email { get; set; }
                public bool Active { get; set; }
                public string CreatedDate { get; set; }
                public string[] Roles { get; set; }
            }
    
            [Fact]
            public void WhenGivenJsonObjectThenEntityDTOReturned()
            {
                //{
                //    "Email": "james@example.com",//    "Active": true,//    "CreatedDate": "2013-01-20T00: 00: 00Z",//    "Roles": [
                //    "User",//    "Admin"
                //    ]
                //}
    
                using (var stream = new FileStream(@"c:\temp\json.json",FileMode.Open))
                {
                    var reader = PipeReader.Create(stream,new StreamPipeReaderOptions(bufferSize:4096 *2));
    
                    ValueTask<ReadResult> readResult = reader.ReadAsync();
                    ReadOnlySequence<byte> buffer = readResult.Result.Buffer;
                    Assert.True(readResult.IsCompleted);
                    var jsonStreamReader = new Utf8JsonReader(buffer);
                    var expectedJson = JsonSerializer
                        .Deserialize<Person>(Encoding.UTF8.GetString(buffer.ToArray()));
                    Assert.Equal("james@example.com",expectedJson.Email);
                }
            }
,

您可以通过PipeReader以这种方式读取文件-

            using (var stream = new FileStream(@"test.json",FileMode.Open))
            {
                var pipeReader = System.IO.Pipelines.PipeReader.Create(stream);

                while (true)
                {
                    var pipeReadResult = await pipeReader.ReadAsync();
                    var buffer = pipeReadResult.Buffer;

                    try
                    {
                        //process data in buffer
                        Console.WriteLine(buffer.Length.ToString());

                        if (pipeReadResult.IsCompleted)
                        {
                            break;
                        }
                    }
                    finally
                    {
                        pipeReader.AdvanceTo(buffer.End);
                    }
                }
            }

如果将PipeReader的缓冲区大小设置为大于要读取的文件大小,则无需循环,但这超出了使用PipeReader的目的-逐段动态处理数据。 / p>

我认为PipeReader不适合您的情况。 Utf8JsonReader可以直接使用文件流。我认为这就是您需要的-

https://docs.microsoft.com/en-us/dotnet/standard/serialization/write-custom-serializer-deserializer#read-from-a-stream-using-utf8jsonreader