通过TCP连接发送大文件

7
我需要通过互联网向一台计算机发送几个大文件。因此,我在路由器上打开了我计划使用的端口,并转发了相应的IP地址。无论如何,让我向您展示我一直在努力实现这一目标的类。这些类可以很好地处理小文件,但有时在处理大文件时会失败。
以下是服务器端的代码:(它是一个控制台应用程序)
using System;
using System.IO;
using System.Net;
using System.Net.Sockets;

namespace ConsoleApplication20
{

    //server
    class Program
    {
        static Server s;
        public static void mm()
        {
            s = new Server("192.168.0.196");
            s.startServer();
            Console.Read();
        }

        static void Main(string[] args)
        {
           // Thread t = new Thread(new ThreadStart(mm));
           // t.Start();
            mm();
            Console.Read();
            s.disconnect();
        }


    }



    class MyTCP
    {
        protected const int MaxChunkSize = 4096;

        protected Int32 port { get; set; }
        protected string serverIP { get; set; }
        protected TcpClient client { get; set; }
        protected static NetworkStream stream { get; set; }

        protected void sendData(NetworkStream stream, Byte[] data)
        {
            // Send the message to the connected TcpServer. 
            stream.Write(data, 0, data.Length);
        }

        protected String receiveData(NetworkStream stream)
        {
            // Buffer to store the response bytes.
            Byte[] data = new Byte[MaxChunkSize];

            // String to store the response ASCII representation.
            String responseData = String.Empty;

            // Read the first batch of the TcpServer response bytes.
            Int32 bytes = stream.Read(data, 0, data.Length);
            responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
            Console.WriteLine("Received: {0}", responseData);

            return responseData;
        }

        protected static Byte[] textToBytes(string text)
        {
            return System.Text.Encoding.ASCII.GetBytes(text);
        }

        public virtual void disconnect() { }

        public bool isServerConected { get { return client.Connected; } }
    }

    [Serializable]
    public class FileProperties 
    {
        public string FileName { get; set; }
        public string DestPath { get; set; }
        public double FileSize { get; set; }

        public FileAttributes fileAttributes { get; set; }
        public System.Security.AccessControl.FileSecurity FileSecurity { get; set; }
        public DateTime creationTime { get; set; }
        public DateTime lastAccessTime { get; set; }
        public DateTime lastWriteTime { get; set; }   
    }

    class Server: MyTCP
    {
        private System.IO.FileStream _FileStream;
        private static TcpListener server;
        private static bool disconect;

        /// <summary>
        /// Constructor
        /// </summary>
        /// <param name="localAddr">The ip address of the server</param>
        /// <param name="port">on what port the server going to be listening to?</param>
        /// <param name="autoStartServer">start listening for connections now? you may call the startserver() method latter...</param>
        public Server(string localAddr, Int32 port = 13000, bool autoStartServer = false)
        {
            this.port = port;
            this.serverIP = localAddr;

            if (autoStartServer)
                start();
        }

        /// <summary>
        /// Start listening for connections
        /// </summary>
        public void startServer()
        {
            start();
        }

        public override void disconnect()
        {
            // Close everything.
            stream.Close();
            client.Close();
            server.Stop();
            disconect = true;
        }


        void start()
        {
            server = null;

            try
            {
                // TcpListener server = new TcpListener(port);
                server = new TcpListener(IPAddress.Parse(serverIP), port);

                // Start listening for client requests.
                server.Start();

                // Buffer for reading data
                Byte[] bytes = new Byte[MaxChunkSize];
                String data = null;

                // Enter the listening loop.
                while (disconect==false)
                {
                    Console.Write("Waiting for a connection... ");

                    // Perform a blocking call to accept requests.
                    // You could also user server.AcceptSocket() here.
                    client = server.AcceptTcpClient();
                    Console.WriteLine("Connected!");




                    // Get a stream object for reading and writing
                    stream = client.GetStream();



                    int i;
                    try
                    {
                        // Loop to receive all the data sent by the client.
                        while ((i = stream.Read(bytes, 0, bytes.Length)) != 0)
                        {


                            // Translate data bytes to a ASCII string.
                            data = System.Text.Encoding.ASCII.GetString(bytes, 0, i);
                            Console.WriteLine("Received: {0}", data);

                            if (data.ToUpper().Contains("<sendFile>".ToUpper()))
                            {
                                receiveFile(bytes);
                            }



                            continue;


                        }
                    }
                    catch { }

                    // Shutdown and end connection
                    client.Close();
                }
            }
            catch (SocketException e)
            {
                Console.WriteLine("SocketException: {0}", e);
            }
            finally
            {
                // Stop listening for new clients.
                server.Stop();
            }


            Console.WriteLine("\nHit enter to continue...");
            Console.Read();
        }


        void receiveFile(Byte[] bytes)
        {
            // send 1
            sendData(stream, textToBytes("<1>"));

            // receive 2
            int length = stream.Read(bytes, 0, bytes.Length);
            byte[] tempA = new byte[length];
            for (int k = 0; k < length; k++)
                tempA[k] = bytes[k];

            Stream ms = new MemoryStream(tempA);
            FileProperties p = new FileProperties();
            System.Xml.Serialization.XmlSerializer x = new System.Xml.Serialization.XmlSerializer(p.GetType());

            try
            {
                p = (FileProperties)x.Deserialize(ms);

                if (Directory.Exists(p.DestPath))
                {
                    //send 3
                    sendData(stream, textToBytes("<3>"));
                }
                else
                {
                    //send 3
                    sendData(stream, textToBytes("<no>"));
                    return;
                }
            }
            catch
            {
                //send 3
                sendData(stream, textToBytes("<no>"));
                return;
            }



            int i;

            string temp = Path.Combine(new string[]{ p.DestPath, p.FileName + ".temp"});

            _FileStream = new System.IO.FileStream(temp, System.IO.FileMode.Create, System.IO.FileAccess.Write);

                while ((i = stream.Read(bytes, 0, bytes.Length)) != 0)
                {
                    if (i == 11 & System.Text.Encoding.ASCII.GetString(bytes, 0, i).ToUpper().Equals("</sendFile>".ToUpper()))
                    {
                        _FileStream.Close();

                        Console.WriteLine("D!");

                        File.SetAttributes(temp, p.fileAttributes);
                        File.SetAccessControl(temp, p.FileSecurity);
                        File.SetCreationTime(temp, p.creationTime);
                        File.SetLastAccessTime(temp, p.lastAccessTime);
                        File.SetLastWriteTime(temp, p.lastWriteTime);

                        if(File.Exists(temp.Substring(0, temp.Length - 4)))
                            File.Delete(temp.Substring(0, temp.Length - 4));

                        File.Move(temp, temp.Substring(0, temp.Length - 4));


                        //sendData(stream, textToBytes("<done>"));

                        Console.WriteLine("Done!");

                        return;
                    }
                    _FileStream.Write(bytes, 0, i);

                }



            return;

        }
    }
}

我的客户端代码如下:

using System;
using System.Net.Sockets;
using System.Windows;
using System.IO;


namespace WpfApplication23sdfd
{

    [Serializable]
    public class FileProperties 
    {
        public string FileName { get; set; }
        public string DestPath { get; set; }
        public double FileSize { get; set; }

        public FileAttributes fileAttributes { get; set; }
        public System.Security.AccessControl.FileSecurity FileSecurity { get; set; }
        public DateTime creationTime { get; set; }
        public DateTime lastAccessTime { get; set; }
        public DateTime lastWriteTime { get; set; }
    }

    abstract class MyTCP
    {
        protected const int MaxChunkSize = 4096;

        protected Int32 port { get; set; }
        protected string serverIP { get; set; }
        protected TcpClient client { get; set; }
        protected static NetworkStream stream { get; set; }

        protected void sendData(NetworkStream stream, Byte[] data)
        {

            // Send the message to the connected TcpServer. 
            stream.Write(data, 0, data.Length);

            // Receive the TcpServer.response.
        }

        protected String receiveData(NetworkStream stream)
        {
            // Buffer to store the response bytes.
            Byte[] data = new Byte[MaxChunkSize];

            // String to store the response ASCII representation.
            String responseData = String.Empty;

            // Read the first batch of the TcpServer response bytes.
            Int32 bytes = stream.Read(data, 0, data.Length);
            responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
            Console.WriteLine("Received: {0}", responseData);

            return responseData;
        }

        protected static Byte[] textToBytes(string text)
        {
            return System.Text.Encoding.ASCII.GetBytes(text);
        }

        public virtual void disconnect() { }

        public bool isServerConected { get { return client.Connected; } }
    }

    //client
    class Client: MyTCP
    {



        /// <summary>
        /// Constructor
        /// </summary>
        /// <param name="serverIP">the ip address of the server</param>
        /// <param name="port">through what port is the connection going to be established</param>
        public Client(string serverIP, Int32 port = 13000, bool autoConnect = false)
        {
            this.port = port;
            this.serverIP = serverIP;

            if (autoConnect)
                connect();
        }


        public bool connect()
        {
            Byte[] data = System.Text.Encoding.ASCII.GetBytes("connect");

            // Create a TcpClient.
            // Note, for this client to work you need to have a TcpServer 
            // connected to the same address as specified by the server, port
            // combination.
            try
            {
                client = new TcpClient(serverIP, port);

                // Get a client stream for reading and writing.
                //  Stream stream = client.GetStream();
                stream = client.GetStream();

                return true;


            }
            catch
            {
                return false;
            }

        }

        public override void disconnect()
        {
            // Close everything.
            stream.Close();
            client.Close();
        }

        static void ConnectOld(String server, Byte[] data)
        {
            try
            {
                // Create a TcpClient.
                // Note, for this client to work you need to have a TcpServer 
                // connected to the same address as specified by the server, port
                // combination.
                Int32 port = 13000;
                TcpClient client = new TcpClient(server, port);



                // Get a client stream for reading and writing.
                //  Stream stream = client.GetStream();

                NetworkStream stream = client.GetStream();

                // Send the message to the connected TcpServer. 
                stream.Write(data, 0, data.Length);



                // Receive the TcpServer.response.

                // Buffer to store the response bytes.
                data = new Byte[256];

                // String to store the response ASCII representation.
                String responseData = String.Empty;

                // Read the first batch of the TcpServer response bytes.
                Int32 bytes = stream.Read(data, 0, data.Length);
                responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
                Console.WriteLine("Received: {0}", responseData);

                // Close everything.
                stream.Close();
                client.Close();
            }
            catch (ArgumentNullException e)
            {
                Console.WriteLine("ArgumentNullException: {0}", e);
            }
            catch (SocketException e)
            {
                Console.WriteLine("SocketException: {0}", e);
            }

            Console.WriteLine("\n Press Enter to continue...");
            Console.Read();
        }

        public void sendFile(string file, string destPath = "c:\\")
        {

            //let server know what you are going to be doing...
            sendData(stream, textToBytes("<sendFile>"));

            FileProperties p = new FileProperties { 
                creationTime = File.GetCreationTime(file), 
                fileAttributes = File.GetAttributes(file), 
                FileSecurity = File.GetAccessControl(file), 
                lastAccessTime = File.GetLastAccessTime(file), 
                lastWriteTime = File.GetLastWriteTime(file),
                 DestPath = destPath,
                 FileName = Path.GetFileName(file)
            };


            // receive 1
            if (!receiveData(stream).ToUpper().Contains("<1>".ToUpper()))
            {
                MessageBox.Show("Error comunicating with server");
                return;
            }

            // send object p to server
            System.Xml.Serialization.XmlSerializer x = new System.Xml.Serialization.XmlSerializer(p.GetType());
            x.Serialize(stream, p); // send 2

            //recieve 3
            if (!receiveData(stream).ToUpper().Contains("<3>".ToUpper()))
            {
                MessageBox.Show("Error incorrect parameters sent to server");
                return;
            }


            System.IO.FileStream streamFile = new System.IO.FileStream(file, System.IO.FileMode.Open, System.IO.FileAccess.Read);



            while (true)
            {
                byte[] chunk = new byte[MaxChunkSize];

                int index = 0;
                // There are various different ways of structuring this bit of code.
                // Fundamentally we're trying to keep reading in to our chunk until
                // either we reach the end of the stream, or we've read everything we need.
                while (index < chunk.Length)
                {
                    int bytesRead = streamFile.Read(chunk, index, chunk.Length - index);


                    if (bytesRead == 0)
                    {
                        break;
                    }
                    if (bytesRead < MaxChunkSize)
                    {
                        byte[] temp = new byte[bytesRead];

                        for (var i = 0; i < bytesRead; i++)
                            temp[i] = chunk[i];

                        chunk = temp;
                    }


                    index += bytesRead;
                }
                if (index != 0) // Our previous chunk may have been the last one
                {
                    sendData(stream,chunk); // index is the number of bytes in the chunk
                }
                if (index != chunk.Length) // We didn't read a full chunk: we're done
                {

                    sendData(stream, textToBytes("</sendFile>".ToUpper()));

                    //receiveData(stream);//wait recall missing to check results

                    return;
                }
            }

        }

    }
}

我实例化客户端类的方式是提供服务器的IP地址:

Client c = new Client("192.168.0.196");
c.sendFile(@"A:\Users\Tono\Desktop\a.mp4");

在执行该代码之前,服务器必须先运行。

我不知道为什么使用套接字发送文件到互联网上如此复杂。我不了解WCF,因此我花费了很多时间创建这些类。也许已经有一些内置的类可以让我将文件发送到另一台计算机上。我只知道网络的基础知识,因此如果能用一个简单的类就好了。我不明白为什么我的类不总是有效?如果我增加缓冲区大小,我的类会更有效吗?在发送更多字节之前,我是否需要等待或暂停程序一段时间?如果有人能告诉我这些类的问题所在就太好了。它们对小文件的处理效果很好,但对大文件有时候不起作用...


考虑使用PacketProtocol https://blog.stephencleary.com/2009/04/sample-code-length-prefix-message.html - oo_dev
4个回答

13

这是我发送大文件的代码。 一些提示:

  • 检查您的缓冲区大小。如果太大,它会失败。
  • 套接字标志。部分标志效果最好。
  • 由于传输时间,需要设置套接字超时。

客户端:

string IPAddress = "";
        int Port = 500;

        string Filename = @"C:\Users\Ben\Desktop\TT.zip";


        int bufferSize = 1024;
        byte[] buffer = null;
        byte[] header = null;


        FileStream fs = new FileStream(Filename, FileMode.Open);
        bool read = true;

        int bufferCount = Convert.ToInt32(Math.Ceiling((double)fs.Length / (double)bufferSize));



        TcpClient tcpClient = new TcpClient(IPAddress, Port);
        tcpClient.SendTimeout = 600000;
        tcpClient.ReceiveTimeout = 600000;

        string headerStr = "Content-length:" + fs.Length.ToString() + "\r\nFilename:" + @"C:\Users\Administrator\Desktop\" + "test.zip\r\n";
        header = new byte[bufferSize];
        Array.Copy(Encoding.ASCII.GetBytes(headerStr), header, Encoding.ASCII.GetBytes(headerStr).Length);

        tcpClient.Client.Send(header);

        for (int i = 0; i < bufferCount; i++)
        {
            buffer = new byte[bufferSize];
            int size = fs.Read(buffer, 0, bufferSize);

            tcpClient.Client.Send(buffer,size,SocketFlags.Partial);

        }

        tcpClient.Client.Close();

        fs.Close();

服务器:

int Port = 500;

        TcpListener listener = new TcpListener(IPAddress.Any, Port);
        listener.Start();


        Socket socket = listener.AcceptSocket();

        int bufferSize = 1024;
        byte[] buffer = null;
        byte[] header = null;
        string headerStr = "";
        string filename = "";
        int filesize = 0;


        header = new byte[bufferSize];

        socket.Receive(header);

        headerStr = Encoding.ASCII.GetString(header);


        string[] splitted = headerStr.Split(new string[] { "\r\n" }, StringSplitOptions.None);
        Dictionary<string, string> headers = new Dictionary<string, string>();
        foreach (string s in splitted)
        {
            if (s.Contains(":"))
            {
                headers.Add(s.Substring(0,s.IndexOf(":")), s.Substring(s.IndexOf(":") + 1));
            }

        }
        //Get filesize from header
        filesize = Convert.ToInt32(headers["Content-length"]);
        //Get filename from header
        filename = headers["Filename"];

        int bufferCount = Convert.ToInt32(Math.Ceiling((double)filesize / (double)bufferSize));


        FileStream fs = new FileStream(filename, FileMode.OpenOrCreate);

        while(filesize > 0)
        {
            buffer = new byte[bufferSize];

            int size = socket.Receive(buffer,SocketFlags.Partial);

            fs.Write(buffer,0,size);

            filesize -= size;
        }


        fs.Close();

希望这能帮助到某人。


+1 谢谢,这非常有用。请注意,在使用服务器接收头文件时,必须先删除尾随的“\0”。我使用了(header.Replace("\0", string.Empty))。 - mahmoud nezar sarhan
如果缓冲区大小太大,为什么会失败? - Altiano Gerung
如果我想让客户端从服务器下载文件,您的示例是否有效?我有点新手,正在尝试修改您的示例以获得该行为(从服务器同步文件到客户端)。谢谢! - coolfarmer
这段代码不太好。如果连接速度慢,服务器可能会在头部接收到文件的一部分,从而在写入实际文件时丢失该部分! - Tono Nam

10

我能立即看到几个问题。其中可能导致您的程序仅在某些时候工作的原因是通过TCP发送不保证每个send都会在另一侧产生完全相同大小的receive

您的协议似乎假定它将这样做,因为您正在等待对于</sendFile>正好为11字节的读取,而它可以在多个独立读取中接收。例如:"[文件数据......]”。如果发生这种情况,您的代码将无法正确完成。

值得注意的是,ASCII编码是7位,因此二进制文件(如MP4)将被错误接收(即使您修复了上面的问题)。如果它是二进制数据,则不应尝试将其转换为字符串,而是直接从byte[]写入文件。

如果您希望继续使用此路线(而不是使用另一个回答中提到的现有的许多文件传输系统),则还可以更改您的协议,以便不是用<sendFile>...</sendFile>来分隔文件,而是最初发送文件长度,这将允许您发送可能包含这些特殊标记之一的文件。


非常感谢。我的问题在于你所说的第二段。我需要找到一种方法让服务器知道传输已经完成,以便它可以开始监听其他命令。此外,我将每个文件分成块,因为当将大文件放入字节数组时会出现内存错误。 - Tono Nam
我的代码没有很好的组织...我没有将ASCII字符写入我正在传输的文件 :) - Tono Nam
现在铱已经可以工作了。在发送“</sendFile>”之前,我等待了几秒钟,并且还增加了块大小,现在它运行得很好。我怎么知道流是否正在使用或客户端是否正在发送字节?如果我能知道这一点,那就太好了,这样我就不必等待几秒钟才发送“</sendFile>”,而是等待流处于“空闲”状态以便发送“</sendFile>”,从而使我的代码更加高效... - Tono Nam

5

通过套接字发送文件/目录并非易事。建议使用一些文件传输库(通过套接字,或者更高级的协议,例如rsync、ftp、http等),而不是试图从头开始编写代码。

在浏览代码后,尝试发送一个包含一些统一内容(例如填充为“0”的内容)的大文件。如果它顺利通过,那么你的xml代码就不起作用了。


是的,你说得对,我会开始寻找一些库。XML序列化可以节省我的时间。它不仅可以避免传递多个字符串,还能够传递一个对象。 - Tono Nam

2

你现在做的基本上是正确的。我能提供的主要建议是将MaxChunkSize设置为较大的值,最好是65000。这样可以使套接字代码处理任何分段,比你自己分段更有效。

此外,你应该意识到发送大文件需要一些时间。在100 Mbit LAN上,理论带宽为每秒12.5兆字节。因此,发送一个700 MByte的文件仍然需要56秒。当然,实际吞吐量取决于网络、计算机和网络硬件中的许多因素,所以速度会慢一些。

最后,Wireshark(或任何其他嗅探程序)是网络编程人员工具箱中最基本且最有价值的工具。在服务器和客户端上使用它来查看TCP数据包的传输方式,以查看是否可以确定缓慢传输的原因。


网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接