如何在.NET中将网页下载到流中

Question

我知道这应该是一个基本问题，但我一直遇到困难。 我想去一个URL/URI下载结果字符串，就像打开文件一样，然后将其放入一个字符串变量中。

我一直在尝试使用IO.Stream和Net.httpxxx，但是还没有成功地使元素正确对齐。

当我在标准流中打开页面时，会收到“不支持给定路径格式”的错误提示，因为它不在本地文件系统中...我理解这一点，但我不明白的是...如何实现等效于：

Public Function GetWebPageAsString(pURL As String) As String
        Dim lStream As IO.StreamReader = New System.IO.StreamReader(pURL)
        Return lStream.ReadToEnd

End Function

- Robin Vessey

---

谢谢大家，每个解决方案最终都起作用了，我选择的那个返回了我想要的结果，而不是我要求的结果，这就是我选择它的原因。（而且它只有两行VB代码）。下面是VB程序员的翻译： Dim client As System.Net.WebClient = New System.Net.WebClient() Dim html As String = client.DownloadString("http://www.google.com") - Robin Vessey

额外说明：我刚意识到我的问题的一半是我正在尝试获取的URL具有“-”，在进行请求之前需要对其进行转义。如果昨晚半夜我能意识到这一点，我可能就不需要问了 :) - Robin Vessey

Answer 1

简短的回答，在C#中看起来像这样：

using(System.Net.WebClient client = new System.Net.WebClient())
{
  string html = client.DownloadString("http://www.google.com");
}

- H H

---

但是这并没有给您一个流（Stream），正如 OP 所要求的那样。 - M4N

@Martin：它包括了一个解决“然后将其放入字符串变量中”的部分的解决方案。 - H H

严格来说，这不是我要求的，但却是我想要的 :) - Robin Vessey

Answer 2

WebClient.OpenRead()可能是你要找的。

以下是来自上述MSDN页面的示例:

 Dim uriString as String
 uriString = "http://www.google.com"

 Dim myWebClient As New WebClient()

 Console.WriteLine("Accessing {0} ...", uriString)

 Dim myStream As Stream = myWebClient.OpenRead(uriString)

 Console.WriteLine(ControlChars.Cr + "Displaying Data :" + ControlChars.Cr)
 Dim sr As New StreamReader(myStream)
 Console.WriteLine(sr.ReadToEnd())

 myStream.Close()

- M4N

---

谢谢，但我得到了“给定路径格式不受支持”的错误提示，在以下代码行上： Dim myStream As Stream = myWebClient.OpenRead(uriString)这是针对网页http://www.alllotto.com/Arizona-Pick-3-May-2009-Lottery-Results.php 的问题吗？ - Robin Vessey

与这个页面相同的是https://dev59.com/kEfSa4cB1Zd3GeqPAMxY也会出现同样的错误，这正是我整晚以来一直在遇到的问题。 - Robin Vessey

当我使用此问题的URL或“http://www.google.com”运行上述示例时，它按预期工作。您能否将完整的代码添加到问题中？ - M4N

我已经更新了代码，并添加了一个示例URL。这对我有效。 - M4N

Answer 3

这个函数可以将任何URI下载到文件中。您可以轻松地将其调整为放入字符串变量中：

public static int DownloadFile(String remoteFilename, String localFilename, bool enforceXmlSafe)
{
// Function will return the number of bytes processed
// to the caller. Initialize to 0 here.
int bytesProcessed = 0;

// Assign values to these objects here so that they can
// be referenced in the finally block
Stream remoteStream = null;
Stream localStream = null;
WebResponse response = null;            

// Use a try/catch/finally block as both the WebRequest and Stream
// classes throw exceptions upon error
try
{
    // Create a request for the specified remote file name
    WebRequest request = WebRequest.Create(remoteFilename);
    if (request != null)
    {
        // Send the request to the server and retrieve the
        // WebResponse object 
        response = request.GetResponse();
        if (response != null)
        {
            // Once the WebResponse object has been retrieved,
            // get the stream object associated with the response's data
            remoteStream = response.GetResponseStream();

            // Create the local file
            if (localFilename != null)
                localStream = File.Create(localFilename);
            else
                localStream = new MemoryStream();

            // Allocate a 1k buffer
            byte[] buffer = new byte[1024];
            int bytesRead;

            // Simple do/while loop to read from stream until
            // no bytes are returned
            do
            {
                // Read data (up to 1k) from the stream
                bytesRead = remoteStream.Read(buffer, 0, buffer.Length);

                // Write the data to the local file
                localStream.Write(buffer, 0, bytesRead);

                // Increment total bytes processed
                bytesProcessed += bytesRead;
            } while (bytesRead > 0);
        }
    }
}
catch (Exception e)
{
    Console.WriteLine(e.Message);
}
finally
{
    // Close the response and streams objects here 
    // to make sure they're closed even if an exception
    // is thrown at some point
    if (response != null) response.Close();
    if (remoteStream != null) remoteStream.Close();
    if (localStream != null) localStream.Close();
}

// Return total bytes processed to caller.
return bytesProcessed;
}

- JerSchneid