我有一个Table Blob,它有一个varbinary(max)作为列.现在我想使用Filestream将数据存储到数据库中.数据可能非常大(在我的情况下为1.5GB),所以我不想将整个数据加载到缓冲区中.
我尝试了什么:
using (FileStream fs = File.Open(@"BigData.iso",FileMode.Open)) { using (sqlConnection conn = new sqlConnection()) { conn.ConnectionString = @"..."; conn.open(); sqlCommand command = new sqlCommand("INSERT INTO Blob Values (@0,@1)",conn); command.Parameters.Add(new sqlParameter("0",Guid.NewGuid())); var sqlb = new sqlBytes(fs); command.Parameters.Add(new sqlParameter("1",sqlDbType.VarBinary,-1)).Value = sqlb; command.ExecuteNonQuery(); } }
但是我得到了一个OutOfMemoryException,因为sqlBytes将其缓冲区初始化为整个数据大小.
有没有办法实现这个目标?
解决方法
您可以以小块的形式读取文件并将它们附加到数据列.
您将需要一个IDENTITY列或另一列,可用作执行UPDATE语句的键.以下是使用IDENTITY列的示例:
创建一个表来存储数据
CREATE TABLE [dbo].[table1]( [ID] [int] IDENTITY(1,1) PRIMARY KEY NOT NULL,[Data] [varbinary](max) NULL,)
实现C#以块的形式插入/更新数据
private const string C_sqlConnectionString = @"Server=SERVERNAME;Database=dbnAME;Trusted_Connection=yes;"; private const int C_FileChunkSizeBytes = 1024 * 1024; // 1 MB private static void storeFile(string filepath) { using (FileStream fs = File.Open(filepath,FileMode.Open)) { using (sqlConnection conn = new sqlConnection()) { conn.ConnectionString = C_sqlConnectionString; conn.open(); // Use a transaction to ensure that all parts of the file get stored to DB sqlCommand command = new sqlCommand("BEGIN TRAN",conn); command.ExecuteNonQuery(); var pos = 0; byte[] fileBytes = null; int sqlRowId = 0; // Read the file in chunks while (pos < fs.Length) { // Read file bytes var bytesToRead = pos + C_FileChunkSizeBytes < fs.Length ? C_FileChunkSizeBytes : (int)(fs.Length - pos); fileBytes = new byte[bytesToRead]; fs.Read(fileBytes,bytesToRead); // Store bytes to a parameter var varbinary = new sqlParameter("0",System.Data.sqlDbType.VarBinary,-1); varbinary.Value = fileBytes; if (pos == 0) { // If this is the first chunk,then we need to INSERT // The HOLDLOCK hint will hold a lock on the table until transaction completes (or is rolled back) command = new sqlCommand("INSERT INTO [dbo].[table1] WITH(HOLDLOCK) VALUES(@0)",conn); command.Parameters.Add(varbinary); command.ExecuteNonQuery(); // Get the row ID for the inserted row command = new sqlCommand("SELECT @@IDENTITY",conn); sqlRowId = Convert.ToInt32(command.ExecuteScalar()); } else { // Update existing row and append data command = new sqlCommand("UPDATE [dbo].[table1] SET [Data] = [Data] + @0 WHERE [ID] = @1",conn); command.Parameters.Add(varbinary); command.Parameters.Add(new sqlParameter("1",System.Data.sqlDbType.Int)).Value = sqlRowId; command.ExecuteNonQuery(); } // ** Good place for a breakpoint pos += bytesToRead; } // Commit transaction command = new sqlCommand("COMMIT TRAN",conn); command.ExecuteNonQuery(); conn.Close(); } } }
测试
在while循环底部的C#代码中放置一个断点,例如pos = bytesToRead;.
在执行代码时,当代码执行在断点处停止时,检查sql中的数据:
SELECT *,LEN([Data]) AS [Length] FROM [dbo].[table1] WITH(NOLOCK)
NOLOCK提示将让我们看到未提交的事务中的数据. LEN([Data])将显示每次迭代while循环后字段长度的增长情况.