从FTP服务器下载大文件（~150MB）时卡住了

Question

我试图使用以下代码从FTP服务器下载文件：

using (System.IO.FileStream fileStream = System.IO.File.OpenWrite(filePath))
              {

                byte[] buffer = new byte[4096];

                int bytesRead = responseStream.Read(buffer, 0, 4096);
                while (bytesRead > 0)
                {
                  fileStream.Write(buffer, 0, bytesRead);                      
                  bytesRead = responseStream.Read(buffer, 0, 4096);
                }

              }

创建responseStream：

System.IO.Stream responseStream = GetFileAsStream(url, username, password, false);

public static System.IO.Stream GetFileAsStream(string ftpUrl, string username, string password, bool usePassive)
{
  System.Net.FtpWebRequest request = (System.Net.FtpWebRequest)System.Net.WebRequest.Create(ftpUrl);
  request.KeepAlive = false;
  request.ReadWriteTimeout = 120000;
  request.Timeout = -1;
  request.UsePassive = usePassive;

  request.Credentials = new System.Net.NetworkCredential(username, password);

  request.Method = System.Net.WebRequestMethods.Ftp.DownloadFile;

  System.IO.Stream fileResponseStream;


  System.Net.FtpWebResponse fileResponse = (System.Net.FtpWebResponse)request.GetResponse();



  fileResponseStream = fileResponse.GetResponseStream();

  return fileResponseStream;
}

对于较小的文件，它可以正常工作，但当文件更大时（例如150mb），该过程会挂起。由于某种原因，程序无法理解已完成下载，并仍然尝试读取更多字节。

我更喜欢不包含使用外部库的答案。谢谢

- reederz

---

响应流是如何创建的？ - Oliver

你尝试过其他的FTP服务器吗？ - qehgt

@digitalez 这可能是FTP服务器的问题，而不是您的客户端代码的问题。 - qehgt

@voltagex 没有错误，它只是卡住了。 - reederz

@digitalez 尝试在客户端挂起时使用调试器中断，看看代码执行到哪里了。 - Adam Baxter

Answer 1

我通过引入请求超时来解决了我的问题-如果达到请求超时时间，则程序将抛出WebException。在这种情况下，程序将从离开的地方恢复下载。以下是我的代码：

这是一个方法内部，如果文件已下载则返回true，否则返回false。

Digitalez.DirectoryUtil.EnsureDirectoryExists(relativePath);

        string filePath = System.IO.Path.Combine(relativePath, fileInfo.Name);
        long length = Digitalez.FtpUtil.GetFileLength(fileInfo.FullPath, userName, password, usePassive);
        long offset = 0;
        int retryCount = 10;
        int? readTimeout = 5 * 60 * 1000; //five minutes

        // if the file exists, do not download it
        if (System.IO.File.Exists(filePath))
        {
          return false;
        }

        while (retryCount > 0)
        {

          using (System.IO.Stream responseStream = Captator.Eifos.Net.FtpUtil.GetFileAsStream(fileInfo.FullPath, userName, password, usePassive, offset, requestTimeout: readTimeout != null ? readTimeout.Value : System.Threading.Timeout.Infinite))
          {

            using (System.IO.FileStream fileStream = new System.IO.FileStream(filePath, System.IO.FileMode.Append))
            {
              byte[] buffer = new byte[4096];
              try
              {
                int bytesRead = responseStream.Read(buffer, 0, buffer.Length);

                while (bytesRead > 0)
                {
                  fileStream.Write(buffer, 0, bytesRead);

                  bytesRead = responseStream.Read(buffer, 0, buffer.Length);
                }

                return true;
              }
              catch (System.Net.WebException)
              {
                // Do nothing - consume this exception to force a new read of the rest of the file
              }
            }

            if (System.IO.File.Exists(filePath))
            {
              offset = new System.IO.FileInfo(filePath).Length;
            }
            else
            {
              offset = 0;
            }

            retryCount--;

            if (offset == length)
            {
              return true;
            }

          }
        }

Digitalez.FtpUtil:

public static System.IO.Stream GetFileAsStream(string ftpUrl, string username, string password, bool usePassive, long offset, int requestTimeout)
{
  System.Net.FtpWebRequest request = (System.Net.FtpWebRequest)System.Net.WebRequest.Create(ftpUrl);

  request.KeepAlive = false;
  request.ReadWriteTimeout = requestTimeout;
  request.Timeout = requestTimeout;
  request.ContentOffset = offset;
  request.UsePassive = usePassive;
  request.UseBinary = true;

  request.Credentials = new System.Net.NetworkCredential(username, password);

  request.Method = System.Net.WebRequestMethods.Ftp.DownloadFile;

  System.IO.Stream fileResponseStream;

  System.Net.FtpWebResponse fileResponse = (System.Net.FtpWebResponse)request.GetResponse();

  fileResponseStream = fileResponse.GetResponseStream();

  return fileResponseStream;
}

public static long GetFileLength(string ftpUrl, string username, string password, bool usePassive)
{
  System.Net.FtpWebRequest request = (System.Net.FtpWebRequest)System.Net.WebRequest.Create(ftpUrl);

  request.KeepAlive = false;
  request.UsePassive = usePassive;


  request.Credentials = new System.Net.NetworkCredential(username, password);
  request.Method = System.Net.WebRequestMethods.Ftp.GetFileSize;

  System.Net.FtpWebResponse lengthResponse = (System.Net.FtpWebResponse)request.GetResponse();
  long length = lengthResponse.ContentLength;
  lengthResponse.Close();
  return length;

}

我没有尝试过其他服务器，但这个肯定可以解决问题。

- reederz

Answer 2

使用您的代码，我已成功下载了几个超过150MB的文件。正如其他人所建议的那样，这很可能是您的FTP服务器出现问题。

- snwr

---

我几乎确定是这样的，但在这个特定的应用程序中，我必须使用我所提供的服务器。 - reederz