c# – WebRequest无法正常下载大文件(〜1 GB)

我试图从公共URL下载一个大文件.起初似乎工作正常,但是1/10的电脑似乎超时了.我最初的尝试是使用WebClient.DownloadFileAsync,但因为它永远不会完成,我回到使用WebRequest.Create并直接读取响应流.

我的第一个使用WebRequest.Create的版本发现与WebClient.DownloadFileAsync相同的问题.操作超时,文件未完成.

如果下载超时,我的下一个版本添加了重试.这是奇怪的吗下载确实最终完成了1次重试以完成最后的7092个字节.所以文件下载的大小相同,但文件已损坏,与源文件不同.现在我会期望这个腐败是在最后的7092个字节,但情况并非如此.

使用BeyondCompare我发现从损坏的文件中丢失了2个字节,总共丢失了7092个字节！这个丢失的字节在1CA49FF0和1E31F380之间,在下载超时并重新启动之前.

这可能会发生什么？有关如何进一步追踪这个问题的提示？

以下是有关代码.

public void DownloadFile(string sourceUri,string destinationPath)
{
    //roughly based on: https://stackoverflow.com/questions/2269607/how-to-programmatically-download-a-large-file-in-c-sharp
    //not using WebClient.DownloadFileAsync as it seems to stall out on large files rarely for unkNown reasons.

    using (var fileStream = File.Open(destinationPath,FileMode.Create,FileAccess.Write,FileShare.Read))
    {
        long totalBytesToReceive = 0;
        long totalBytesReceived = 0;
        int attemptCount = 0;
        bool isFinished = false;

        while (!isFinished)
        {
            attemptCount += 1;

            if (attemptCount > 10)
            {
                throw new InvalidOperationException("Too many attempts to download. Aborting.");
            }

            try
            {
                var request = (HttpWebRequest)WebRequest.Create(sourceUri);

                request.Proxy = null;//https://stackoverflow.com/questions/754333/why-is-this-webrequest-code-slow/935728#935728
                _log.Addinformation("Request #{0}.",attemptCount);

                //continue downloading from last attempt.
                if (totalBytesReceived != 0)
                {
                    _log.Addinformation("Request resuming with range: {0},{1}",totalBytesReceived,totalBytesToReceive);
                    request.AddRange(totalBytesReceived,totalBytesToReceive);
                }

                using (var response = request.GetResponse())
                {
                    _log.Addinformation("Received response. ContentLength={0},ContentType={1}",response.ContentLength,response.ContentType);

                    if (totalBytesToReceive == 0)
                    {
                        totalBytesToReceive = response.ContentLength;
                    }

                    using (var responseStream = response.GetResponseStream())
                    {
                        _log.Addinformation("Beginning read of response stream.");
                        var buffer = new byte[4096];
                        int bytesRead = responseStream.Read(buffer,buffer.Length);
                        while (bytesRead > 0)
                        {
                            fileStream.Write(buffer,bytesRead);
                            totalBytesReceived += bytesRead;
                            bytesRead = responseStream.Read(buffer,buffer.Length);
                        }

                        _log.Addinformation("Finished read of response stream.");
                    }
                }

                _log.Addinformation("Finished downloading file.");
                isFinished = true;
            }
            catch (Exception ex)
            {
                _log.Addinformation("Response raised exception ({0}). {1}",ex.GetType(),ex.Message);
            }
        }
    }
}

以下是损坏的下载的日志输出：

Request #1.
Received response. ContentLength=939302925,ContentType=application/zip
Beginning read of response stream.
Response raised exception (System.Net.WebException). The operation has timed out.
Request #2.
Request resuming with range: 939295833,939302925
Received response. ContentLength=7092,ContentType=application/zip
Beginning read of response stream.
Finished read of response stream.
Finished downloading file.

解决方法

这是我通常使用的方法,迄今为止,对于您需要的相同类型的加载,它还没有失败.尝试使用我的代码来改变你的一些,看看是否有帮助.

if (!Directory.Exists(localFolder))
{
    Directory.CreateDirectory(localFolder);   
}


try
{
    HttpWebRequest httpRequest = (HttpWebRequest)WebRequest.Create(Path.Combine(uri,filename));
    httpRequest.Method = "GET";

    // if the URI doesn't exist,exception gets thrown here...
    using (HttpWebResponse httpResponse = (HttpWebResponse)httpRequest.GetResponse())
    {
        using (Stream responseStream = httpResponse.GetResponseStream())
        {
            using (FileStream localFileStream = 
                new FileStream(Path.Combine(localFolder,filename),FileMode.Create))
            {
                var buffer = new byte[4096];
                long totalBytesRead = 0;
                int bytesRead;

                while ((bytesRead = responseStream.Read(buffer,buffer.Length)) > 0)
                {
                    totalBytesRead += bytesRead;
                    localFileStream.Write(buffer,bytesRead);
                }
            }
        }
    }
}
catch (Exception ex)
{        
    throw;
}