使用更好的方式读取Excel数据

问题描述

现在我正在从excel中获取数据并遍历各行，并根据条件我正在处理结果，例如将结果存储在对象中以进行进一步处理。

excel工作表约为20 MB，记录数接近7000，我正在使用开放式xml来从excel文件中获取数据，如下面的代码所述。

        string filePath = @"C:\weather-Data\DesignConditions_p.xlsx";
        using FileStream fs = new FileStream(filePath,FileMode.Open,FileAccess.Read,FileShare.ReadWrite);
        using SpreadsheetDocument doc = SpreadsheetDocument.Open(fs,false);
        WorkbookPart workbookPart = doc.WorkbookPart;
        SharedStringTablePart sstpart = workbookPart.GetPartsOfType<SharedStringTablePart>().First();
        SharedStringTable sst = sstpart.SharedStringTable;

        Sheet firstSheet = workbookPart.Workbook.Descendants<Sheet>().First();
        Worksheet sheet = ((WorksheetPart)workbookPart.GetPartById(firstSheet.Id)).Worksheet;

        var rows = sheet.Descendants<Row>();
        var weatherDataList = new List<WeatherStation>();
        foreach (Row row in rows.Skip(5)) // it is taking almost more than 60 minutes to process and enter into the if loop below (country.Equals("USA"))
        {
            var weatherData = new WeatherStation();
            string country = GetCellValue(filePath,"Annual",$"B{row.RowIndex.ToString()}");
            if (country.Equals("USA"))
            {
                weatherData.CountryAbbreviation = country;
                weatherData.StateAbbreviation = GetCellValue(filePath,$"C{row.RowIndex.ToString()}");
                weatherData.Number = GetCellValue(filePath,$"E{row.RowIndex.ToString()}");
                ......
                .......
            }
        }

有人可以指出正确的方向以优化处理时间，同时从excel读取数据时，我正在为此应用程序使用.Net Core

谢谢。

解决方法

您可以使用“ SAX”方法，这样您可以分批读取文件，因此处理和IO可以更快。

// The SAX approach.
    static void ReadExcelFileSAX(string fileName)
    {
        using (SpreadsheetDocument spreadsheetDocument = SpreadsheetDocument.Open(fileName,false))
        {
            WorkbookPart workbookPart = spreadsheetDocument.WorkbookPart;
            WorksheetPart worksheetPart = workbookPart.WorksheetParts.First();

            OpenXmlReader reader = OpenXmlReader.Create(worksheetPart);
            string text;
            while (reader.Read())
            {
                if (reader.ElementType == typeof(CellValue))
                {
                    text = reader.GetText();
                    Console.Write(text + " ");
                }
            }
            Console.WriteLine();
            Console.ReadKey();
        }
    }

https://docs.microsoft.com/en-us/office/open-xml/how-to-parse-and-read-a-large-spreadsheet

除此之外，您可能会希望找到一个读取速度更快的库/ nuget程序包，因为我看不到具有影响力的全部方法来进一步调优该代码。