.NET:是否可能使HttpWebRequest自动解压缩gzip响应?

38
这篇答案中,我描述了如何在HttpWebResponse中的响应流上包装一个GZipStream来进行解压缩。

相关的代码如下:

HttpWebRequest hwr = (HttpWebRequest) WebRequest.Create(url);
hwr.CookieContainer =
    PersistentCookies.GetCookieContainerForUrl(url);
hwr.Accept = "text/xml, */*";
hwr.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip, deflate");
hwr.Headers.Add(HttpRequestHeader.AcceptLanguage, "en-us");
hwr.UserAgent = "My special app";
hwr.KeepAlive = true;

using (var resp = (HttpWebResponse) hwr.GetResponse()) 
{
    using(Stream s = resp.GetResponseStream())
    {
        Stream s2 = s;
        if (resp.ContentEncoding.ToLower().Contains("gzip"))
            s2 = new GZipStream(s2, CompressionMode.Decompress);
        else if (resp.ContentEncoding.ToLower().Contains("deflate"))
            s2 = new DeflateStream(s2, CompressionMode.Decompress);

         ... use s2 ...
    }
}

有没有一种方法可以让HttpWebResponse自动提供解压缩流?换句话说,有没有一种方法可以让我从上面的代码中消除以下内容:

      Stream s2 = s;
      if (resp.ContentEncoding.ToLower().Contains("gzip"))
          s2 = new GZipStream(s2, CompressionMode.Decompress);
      else if (resp.ContentEncoding.ToLower().Contains("deflate"))
          s2 = new DeflateStream(s2, CompressionMode.Decompress);

谢谢。

3个回答

74

按照以下步骤使用HttpWebRequest.AutomaticDecompression属性:

HttpWebRequest hwr = (HttpWebRequest) WebRequest.Create(url);
hwr.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;

当使用该属性时,无需手动添加Accept-Encoding HTTP标头;它将自动添加。

(另外,我知道这只是示例代码,但应将HttpWebResponse对象放置在using块中,以便在完成使用后正确释放它。)


Cha-ching! 谢谢。我怎么会错过那个? - Cheeso
5
"不需要手动添加Accept-Encoding HTTP头" - 此外,如果这样做,头部中的"gzip"和"deflate"值将会重复,可能导致一些服务器拒绝请求。 - Chad

0

我的经验是它已经自动完成了。我必须通过将请求对象的AutomaticDecompression属性设置为DecompressionMethods.None来明确禁用它。


1
嗯,有点奇怪。我需要重新运行我的测试。我认为看到了压缩数据通过。我可能是错的。谢谢。 - Cheeso
1
刚刚测试了一下 - 在我的代码中它没有自动开启。我需要阅读文档来找出原因,但是当我明确启用它时,它消除了我进行解压缩流操作的需要。谢谢你的回答。 - Cheeso
我曾经费了很大的劲才找到一个能按照我的要求压缩数据的网站,http://www.google.com 无法胜任。偶然间,我输入了这个确切的StackOverflow问题的URL,它奏效了——当我将其设置为DecompressionMethods.None(在.NET 4.5.1中显然是默认值)时,我得到了gzip字节。不错。 - BrainSlugs83
根据你所使用的框架版本,其默认设置会有所不同。 - James Westgate

-1

我认为AutomaticDecompression不会添加头部AcceptDecoding=gzip,deflate。 以下是我的请求和响应示例,当我只添加AutomaticDecompression时(未显式添加AcceptEncoding标头),可以看到头部根本没有AcceptEncoding。

//REQUEST 
//===============================================================
-       req {System.Net.HttpWebRequest} System.Net.HttpWebRequest
+       base    {System.Net.HttpWebRequest} System.Net.WebRequest {System.Net.HttpWebRequest}
        _AuthInfo   null    System.Net.ICredentials
        _ContentLength  -1  long
+       _HttpRequestHeaders {User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS .NET CF Web Services Client Protocol 3.5.7283.0)
Authorization: Basic RElOT0I6MTExMQ==

}   System.Net.WebHeaderCollection
        _MaximumAllowedRedirections 50  int
        _MediaType  null    string
+       _OriginUri  {http://192.168.0.106:8084/MyTestApp/Soap/IMyApp}   System.Uri
        _OriginVerb "GET"   string
+       _Proxy  {System.Net.GlobalProxySelection.SystemWebProxy}    System.Net.IWebProxy {System.Net.GlobalProxySelection.SystemWebProxy}
        _ProxyAuthenticationState   null    System.Net.AuthenticationState
        _ReadWriteTimeout   300000  int
        _ServerAuthenticationState  null    System.Net.AuthenticationState
        _Timeout    100000  int
+       _Timer  {System.Threading.Timer}    System.Threading.Timer
+       _Uri    {http://192.168.0.106:8084/MyTestApp/Soap/IMyApp}   System.Uri
        Aborted false   bool
        Accept  Could not evaluate expression   string
+       Address {http://192.168.0.106:8084/MyTestApp/Soap/IMyApp}   System.Uri
        AllowAutoRedirect   false   bool
        AllowWriteStreamBuffering   true    bool
        AutomaticDecompression  GZip | Deflate  System.Net.DecompressionMethods
        ChallengedUri   Could not evaluate expression   System.Uri
+       ClientCertificates  {System.Security.Cryptography.X509Certificates.X509CertificateCollection}   System.Security.Cryptography.X509Certificates.X509CertificateCollection
        ConnectHostAndPort  "192.168.0.106:8084"    string
        Connection  Could not evaluate expression   string
        ConnectionGroupName Could not evaluate expression   string
        ContentLength   -1  long
        ContentType Could not evaluate expression   string
        ContinueDelegate    Could not evaluate expression   System.Net.HttpContinueDelegate
        Credentials Could not evaluate expression   System.Net.ICredentials
+       CurrentAuthenticationState  {System.Net.AuthenticationState}    System.Net.AuthenticationState
        Expect  Could not evaluate expression   string
        hasEntityData   false   bool
        HaveResponse    false   bool
+       Headers {User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS .NET CF Web Services Client Protocol 3.5.7283.0)
Authorization: Basic RElOT0I6MTExMQ==

}   System.Net.WebHeaderCollection
+       IfModifiedSince {9/7/15 5:47:12 AM} System.DateTime
        KeepAlive   true    bool
        m_Aborted   false   bool
        m_allowAutoRedirection  false   bool
        m_allowWriteStreamBuffering true    bool
        m_AutomaticDecompression    GZip | Deflate  System.Net.DecompressionMethods
        m_boundConnection   null    System.Net.Connection
+       m_clientCertificates    {System.Security.Cryptography.X509Certificates.X509CertificateCollection}   System.Security.Cryptography.X509Certificates.X509CertificateCollection
        m_connection    null    System.Net.Connection
        m_ConnectionClosedStopSendingEntityData false   bool
        m_connectionGroupName   null    string
        m_connectionUsers   0   int
        m_connMgrConnection -1  int
        m_continueDelegate  null    System.Net.HttpContinueDelegate
+       m_continueFunction  {System.Threading.TimerCallback}    System.Threading.TimerCallback
+       m_continueTimer null    System.Threading.Timer
        m_doneSendingEvent  null    System.Threading.ManualResetEvent
        m_errorResponse null    System.Net.WebException
        m_expectContinue    false   bool
        m_finishedWriting   false   bool
        m_httpWriteMode None    System.Net.HttpWriteMode
        m_IsCurrentAuthenticationStateProxy false   bool
        m_isSubmitting  false   bool
        m_KeepAlive true    bool
        m_numberRedirections    0   int
        m_Pipelined true    bool
        m_PreAuthenticate   false   bool
+       m_readWriteTimer    {System.Threading.Timer}    System.Threading.Timer
        m_requestGeneration 0   int
+       m_requestSentEvent  {System.Threading.ManualResetEvent} System.Threading.ManualResetEvent
+       m_requestStream null    System.Net.HttpWriteStream
        m_requestStreamRetrieved    false   bool
        m_RequestSubmitted  false   bool
        m_response  null    System.Net.HttpWebResponse
        m_responseComplete  false   bool
+       m_responseEvent {System.Threading.ManualResetEvent} System.Threading.ManualResetEvent
        m_responseRetrieved false   bool
        m_sentHeaders   false   bool
+       m_setProxy  {System.Net.GlobalProxySelection.SystemWebProxy}    System.Net.IWebProxy {System.Net.GlobalProxySelection.SystemWebProxy}
+       m_srvPoint  null    System.Net.ServicePoint
        m_startedReceiving  false   bool
        m_timedOut  0   int
+       m_version   {1.1}   System.Version
        MaximumAutomaticRedirections    50  int
        MediaType   Could not evaluate expression   string
        Method  "GET"   string
        NtlmKeepAlive   false   bool
        Pipelined   true    bool
        PreAuthenticate false   bool
+       ProtocolVersion {1.1}   System.Version
+       Proxy   {System.Net.GlobalProxySelection.SystemWebProxy}    System.Net.IWebProxy {System.Net.GlobalProxySelection.SystemWebProxy}
        ProxyAuthenticatedConnectionSharing false   bool
+       ProxyAuthenticationState    {System.Net.AuthenticationState}    System.Net.AuthenticationState
        ReadWriteTimeout    300000  int
        Referer Could not evaluate expression   string
+       RequestUri  {http://192.168.0.106:8084/MyTestApp/Soap/IMyApp}   System.Uri
+       ResponseStatusCode  'req.ResponseStatusCode' threw an exception of type 'System.NullReferenceException' System.Net.HttpStatusCode {System.NullReferenceException}
        SendChunked false   bool
+       ServerAuthenticationState   {System.Net.AuthenticationState}    System.Net.AuthenticationState
        ServicePoint    Could not evaluate expression   System.Net.ServicePoint
        SyncObject  {object}    object
        Timeout 100000  int
        TransferEncoding    Could not evaluate expression   string
        UserAgent   "Mozilla/4.0 (compatible; MSIE 6.0; MS .NET CF Web Services Client Protocol 3.5.7283.0)"    string
+       UsesProxySemantics  'req.UsesProxySemantics' threw an exception of type 'System.NullReferenceException' bool {System.NullReferenceException}
+       Static members  


//RESPONSE 
=====================================================
-       res {System.Net.HttpWebResponse}    System.Net.HttpWebResponse
+       base    {System.Net.HttpWebResponse}    System.Net.WebResponse {System.Net.HttpWebResponse}
        CharacterSet    ""  string
        ContentEncoding ""  string
        ContentLength   -1  long
        ContentType "text/xml"  string
-       Headers {Date: Mon, 07 Sep 2015 11:49:36 GMT
Server: Apache/2.2.29 (Win32)
Vary: Accept-Encoding,User-Agent
Content-Encoding: 
Content-Length: 1481
Keep-Alive: timeout=60, max=100
Connection: Keep-Alive
Content-Type: text/xml

}   System.Net.WebHeaderCollection
+       base    {Date: Mon, 07 Sep 2015 11:49:36 GMT
Server: Apache/2.2.29 (Win32)
Vary: Accept-Encoding,User-Agent
Content-Encoding: 
Content-Length: 1481
Keep-Alive: timeout=60, max=100
Connection: Keep-Alive
Content-Type: text/xml

}   System.Collections.Specialized.NameValueCollection {System.Net.WebHeaderCollection}
        m_IsHttpWebHeaderObject true    bool
+       Static members      
+       LastModified    {9/7/15 5:49:45 AM} System.DateTime
        m_contentLength -1  long
        m_decompressionMethod   GZip | Deflate  System.Net.DecompressionMethods
-       m_httpResponseHeaders   {Date: Mon, 07 Sep 2015 11:49:36 GMT
Server: Apache/2.2.29 (Win32)
Vary: Accept-Encoding,User-Agent
Content-Encoding: 
Content-Length: 1481
Keep-Alive: timeout=60, max=100
Connection: Keep-Alive
Content-Type: text/xml

}   System.Net.WebHeaderCollection
-       base    {Date: Mon, 07 Sep 2015 11:49:36 GMT
Server: Apache/2.2.29 (Win32)
Vary: Accept-Encoding,User-Agent
Content-Encoding: 
Content-Length: 1481
Keep-Alive: timeout=60, max=100
Connection: Keep-Alive
Content-Type: text/xml

}   System.Collections.Specialized.NameValueCollection {System.Net.WebHeaderCollection}
+       base    {Date: Mon, 07 Sep 2015 11:49:36 GMT
Server: Apache/2.2.29 (Win32)
Vary: Accept-Encoding,User-Agent
Content-Encoding: 
Content-Length: 1481
Keep-Alive: timeout=60, max=100
Connection: Keep-Alive
Content-Type: text/xml

}   System.Collections.Specialized.NameObjectCollectionBase {System.Net.WebHeaderCollection}
        _allKeys    null    string[]
+       AllKeys {string[8]} string[]
        m_IsHttpWebHeaderObject true    bool
+       Static members      
        m_mediaType null    string
        m_method    "POST"  string
-       m_responseStream    {System.Net.GZipWrapperStream}  System.IO.Stream {System.Net.GZipWrapperStream}
+       [System.Net.GZipWrapperStream]  {System.Net.GZipWrapperStream}  System.Net.GZipWrapperStream
        base    {System.Net.GZipWrapperStream}  System.MarshalByRefObject {System.Net.GZipWrapperStream}
        _closeRecursionCounter  0   int
        CanRead true    bool
        CanSeek false   bool
        CanTimeout  false   bool
        CanWrite    false   bool
+       Length  'res.m_responseStream.Length' threw an exception of type 'System.NotSupportedException' long {System.NotSupportedException}
+       Position    'res.m_responseStream.Position' threw an exception of type 'System.NotSupportedException'   long {System.NotSupportedException}
+       ReadTimeout 'res.m_responseStream.ReadTimeout' threw an exception of type 'System.InvalidOperationException'    int {System.InvalidOperationException}
+       WriteTimeout    'res.m_responseStream.WriteTimeout' threw an exception of type 'System.InvalidOperationException'   int {System.InvalidOperationException}
+       Static members      
        m_statusCode    200 int
        m_statusDescription "OK"    string
+       m_url   {http://192.168.0.106:8084/MyTestApp/Soap/IMyApp}   System.Uri
+       m_version   {1.1}   System.Version
        Method  "POST"  string
-       ProtocolVersion {1.1}   System.Version
        _Build  -1  int
        _Major  1   int
        _Minor  1   int
        _Revision   -1  int
        Build   -1  int
        Major   1   int
        Minor   1   int
        Revision    -1  int
-       ResponseUri {http://192.168.0.106:8084/MyTestApp/Soap/IMyApp}   System.Uri
        AbsolutePath    "/MyTestApp/Soap/IMyApp"    string
        AbsoluteUri "http://192.168.0.106:8084/MyTestApp/Soap/IMyApp"   string
        Authority   "192.168.0.106:8084"    string
        DnsSafeHost "192.168.0.106" string
        Fragment    ""  string
        HasAuthority    true    bool
        Host    "192.168.0.106" string
        HostNameType    IPv4    System.UriHostNameType
        HostType    IPv4HostType    System.Uri.Flags
        IsAbsoluteUri   true    bool
        IsDefaultPort   false   bool
        IsDosPath   false   bool
        IsFile  false   bool
        IsImplicitFile  false   bool
        IsLoopback  false   bool
        IsNotAbsoluteUri    false   bool
        IsUnc   false   bool
        IsUncOrDosPath  false   bool
        IsUncPath   false   bool
        LocalPath   "/MyTestApp/Soap/IMyApp"    string
        m_Flags IPv4HostType | AuthorityFound | NotDefaultPort | MinimalUriInfoSet | AllUriInfoSet  System.Uri.Flags
+       m_Info  {System.Uri.UriInfo}    System.Uri.UriInfo
        m_OrigFileString    null    string
        m_String    "http://192.168.0.106:8084/MyTestApp/Soap/IMyApp"   string
+       m_Syntax    {System.UriParser.BuiltInUriParser} System.UriParser {System.UriParser.BuiltInUriParser}
        OriginalString  "http://192.168.0.106:8084/MyTestApp/Soap/IMyApp"   string
        PathAndQuery    "/MyTestApp/Soap/IMyApp"    string
        Port    8084    int
        PrivateAbsolutePath "/MyTestApp/Soap/IMyApp"    string
        Query   ""  string
        Scheme  "http"  string
        SecuredPathIndex    0   ushort
+       Segments    {string[4]} string[]
+       Syntax  {System.UriParser.BuiltInUriParser} System.UriParser {System.UriParser.BuiltInUriParser}
        UserDrivenParsing   false   bool
        UserEscaped false   bool
        UserInfo    ""  string
+       Static members      
        Server  "Apache/2.2.29 (Win32)" string
        StatusCode  OK  System.Net.HttpStatusCode
        StatusDescription   "OK"    string

看起来是这样的 - 请参见此处:http://referencesource.microsoft.com/#System/net/System/Net/HttpWebRequest.cs,4985 - AndersMad
@AndersMad 是的,我看到了你的链接显示AcceptEncoding已经添加,但是当我查看上面显示的标头时,我并没有看到它。 - pixel

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接