C#使用代理的HttpClient

Question

C#使用代理的HttpClient

7

我使用HttpClient执行了许多请求来获取某些资源。为了避免泄漏，我将其作为单个实例使用。现在我想使用代理，那么我该如何为每个请求使用不同的代理呢？

谢谢！

public class Program
{
    private static HttpClient Client = new HttpClient();
    public static void Main(string[] args)
    {
        Console.WriteLine("Starting connections");
        for(int i = 0; i<10; i++)
        {
            var result = Client.GetAsync("http://aspnetmonsters.com").Result;
            Console.WriteLine(result.StatusCode);
        }
        Console.WriteLine("Connections done");
        Console.ReadLine();
    }

}

- Oleksandr Hnatenko

1

为每个代理创建并验证一个HttClient。然后使用与您的请求地址匹配的实例。 - flo scheiwiller

3个回答

3

你需要实现IWebProxy接口。

这里提供了一个非常简单的样例。

首先，实现IWebProxy接口。

public class MyProxy : IWebProxy {
public MyProxy() {  credentials = new NetworkCredential( user, password ); }
private NetworkCredential credentials;
public ICredentials Credentials
{
    get = > credentials;
    set = > throw new NotImplementedException();
}
private Uri proxyUri;
public Uri GetProxy( Uri destination )
{
    return proxyUri; // your proxy Uri
}
public bool IsBypassed( Uri host )
{
    return false;
}
private const string user = "yourusername";
private const string password = "password";}

然后将其提供给HttpClient中的处理程序

public class MyHttpClient {
internal static HttpResult httpMethod( ... )
{
    var _client = client();
    try
    {
        var message = new HttpRequestMessage( method, url );
        message.Content = new StringContent( content, Encoding.UTF8, "application/json" );
        var result = _client.SendAsync( message ).Result;// handle result
    }
    catch( Exception e ){}
}
private static HttpClient client()
{
    var httpClientHandler = new HttpClientHandler() { Proxy = new MyProxy() };
    var httpClient = new MyClient( new Uri( "baseurl" ), httpClientHandler );
    return httpClient;

- Leo

-1

基本上，要能够更改代理，您需要在 HttpClientHandler 上引用。
这里有一个简单的示例：C# use proxy with HttpClient request
还有另一个示例：Simple C# .NET 4.5 HTTPClient Request Using Basic Auth and Proxy

我建议将 HttpClientHandler 保留在私有字段中，并使用引用每次更改代理时都需要它。
请记住，如果您需要同时使用不同的代理，则需要多个 HttpClientHandler 类的实例。

如果您需要我为此编写示例代码，请联系我。

谢谢。

- Nick Polyderopoulos

1

但是在创建之后，你如何将HttpClientHandler设置为HttpClient的实例？ - Sanja Melnichuk

@Sanja Melnichuk，HttpClientHandler 只需要在构造时设置。如果您保留引用，可以对其进行更改，而无需重建 HttpClient。 - Nick Polyderopoulos

所以我有一个静态的httpclient和HttpCleintHandler，我在不同的线程中使用它，每个线程使用自己的代理，如何确保我为即将到来的请求更改代理？ - Sanja Melnichuk

1

这是错误的建议。虽然你可以在发送第一个请求之前更改附加到httpclient的处理程序的代理，但任何后续尝试更改代理都将导致抛出异常。 - cubesnyc

1

@cubesync，首先我不喜欢你的语气。其次，我在我的回答中说过，您可能需要多个实例。至于异常，我以前没有遇到过，所以我会尝试复制并提供解决方案。您是否尝试更改HTTP客户端属性或HTTP客户端处理程序？ - Nick Polyderopoulos

显示剩余3条评论

网页内容由stack overflow 提供, 点击上面的

可以查看英文原文，
原文链接

- Stefan Steiger · Accepted Answer

啊，我误读了问题。
这不是关于如何使用随机的IWebProxy和HttpClientHandler，而是如何解决在第一次请求开始后无法重置同一HttpClientHandler的代理属性的问题。

问题在于你无法重置HttpClientHandler的代理...

``` System.InvalidOperationException: 'This instance has already started one or more requests. Properties can only be modified before sending the first request.' ```

但这还是相当容易的。

- HttpClientHandler的代理属性接受一个实现了IWebProxy的对象。 - IWebProxy接口有一个GetProxy方法，返回代理的Uri。 - 因此，你可以制作自己的类来实现这个接口，并控制它如何通过GetProxy返回代理的Uri。 - 你可以让它包装另一个IWebProxy，在GetProxy中它将返回内部IWebProxy的GetProxy。 - 这样，你就不必改变HttpClientHandler的代理属性，只需改变内部的IWebProxy。

实现：

public class WebProxyService
  : System.Net.IWebProxy
{
    protected System.Net.IWebProxy m_proxy;


    public System.Net.IWebProxy Proxy
    {
        get { return this.m_proxy ??= System.Net.WebRequest.DefaultWebProxy; }
        set { this.m_proxy = value; }
    }

    System.Net.ICredentials System.Net.IWebProxy.Credentials
    {
        get { return this.Proxy.Credentials; }
        set { this.Proxy.Credentials = value; }
    }


    public WebProxyService()
    { } // Constructor 

    public WebProxyService(System.Net.IWebProxy proxy)
    {
        this.Proxy = proxy;
    } // Constructor 


    System.Uri System.Net.IWebProxy.GetProxy(System.Uri destination)
    {
        return this.Proxy.GetProxy(destination);
    }

    bool System.Net.IWebProxy.IsBypassed(System.Uri host)
    {
        return this.Proxy.IsBypassed(host);
    }


}

然后使用方式如下：

public class AlternatingProxy 
{


    public static async System.Threading.Tasks.Task Test()
    {
        string url = "http://aspnetmonsters.com";
       
        System.Net.WebProxy[] proxies = new[] {
            null,
            new System.Net.WebProxy("104.238.172.20", 8080),
            new System.Net.WebProxy("104.238.167.193", 8080),
            new System.Net.WebProxy("136.244.102.38", 8080),
            new System.Net.WebProxy("95.179.202.40", 8080)
        };

        System.Random rnd = new System.Random();
        WebProxyService proxyService = new WebProxyService();
        
        using (System.Net.Http.HttpClient hc = new System.Net.Http.HttpClient(
          new System.Net.Http.HttpClientHandler { UseProxy = true, Proxy = proxyService }
          ))
        {
            // https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent
            hc.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (iPad; CPU OS 12_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148");
            hc.DefaultRequestHeaders.Add("Accept-Language", "fr-FR, fr;q=0.9, en;q=0.8, it;q=0.7, *;q=0.5");
            hc.DefaultRequestHeaders.Add("Referer", "https://www.baidu.com");
            hc.DefaultRequestHeaders.ConnectionClose = true; 

            for (int i = 0; i < 10; ++i)
            {
                proxyService.Proxy = proxies[rnd.Next(proxies.Length)];
                string response = await hc.GetStringAsync(url);
            }
        }

    } // End Task Test  


} // End Class TestMe

编辑：

如果您想使用单例模式，这可能是一个好主意：

public class WebProxyService : System.Net.IWebProxy {

    protected System.Net.IWebProxy m_proxy;


    public System.Net.IWebProxy Proxy
    {
        get { return this.m_proxy ??= System.Net.WebRequest.DefaultWebProxy; }
        set { this.m_proxy = value; }
    }

    System.Net.ICredentials System.Net.IWebProxy.Credentials
    {
        get { return this.Proxy.Credentials; }
        set { this.Proxy.Credentials = value; }
    }


    public WebProxyService()
    { } // Constructor 


    public WebProxyService(System.Net.IWebProxy proxy)
    {
        this.Proxy = proxy;
    } // Constructor 


    protected System.Func<System.Net.WebProxy>[] proxies = new System.Func<System.Net.WebProxy>[] {
            delegate(){ return new System.Net.WebProxy("104.238.172.20", 8080); },
            delegate (){ return new System.Net.WebProxy("104.238.167.193", 8080);},
            delegate(){ return new System.Net.WebProxy("136.244.102.38", 8080);},
            delegate(){ return new System.Net.WebProxy("95.179.202.40", 8080);}
        };


    System.Uri System.Net.IWebProxy.GetProxy(System.Uri destination)
    {
        return proxies[RandomGen2.Next(proxies.Length)]().GetProxy(destination);
    }


    bool System.Net.IWebProxy.IsBypassed(System.Uri host)
    {
        return this.Proxy.IsBypassed(host);
    }



    private static class RandomGen2
    {
        private static System.Random _global = new System.Random();

        [System.ThreadStatic]
        private static System.Random _local;

        public static int Next(int maxValue)
        {
            System.Random inst = _local;
            if (inst == null)
            {
                int seed;
                lock (_global) seed = _global.Next();
                _local = inst = new System.Random(seed);
            }
            return inst.Next(maxValue);
        }
    }


} // End Class WebProxyService

编辑 2：

如果你更改代理，它仍然不是线程安全的。
因此，使用一个固定的不可变代理列表，并阻止设置属性。
这样，它应该是线程安全的。

public class WebProxyService
      : System.Net.IWebProxy
{


    protected System.Net.IWebProxy[] m_proxyList;

    public System.Net.IWebProxy Proxy
    {
        get
        {
            // https://devblogs.microsoft.com/pfxteam/getting-random-numbers-in-a-thread-safe-way/
            if (this.m_proxyList != null)
                return this.m_proxyList[ThreadSafeRandom.Next(this.m_proxyList.Length)];
            
            return System.Net.WebRequest.DefaultWebProxy;
        }
        set
        {
            throw new System.InvalidOperationException("It is not thread-safe to change the proxy-list.");
        }
    }

    System.Net.ICredentials System.Net.IWebProxy.Credentials
    {
        get { return this.Proxy.Credentials; }
        set { this.Proxy.Credentials = value; }
    }


    public WebProxyService()
    {
    } // Constructor 


    public WebProxyService(System.Net.IWebProxy[] proxyList)
    {
        this.m_proxyList = proxyList;
    } // Constructor 


    System.Uri System.Net.IWebProxy.GetProxy(System.Uri destination)
    {
        return this.Proxy.GetProxy(destination);
    }


    bool System.Net.IWebProxy.IsBypassed(System.Uri host)
    {
        return this.Proxy.IsBypassed(host);
    }


} // End Class WebProxyService

Old answer: ----

Using a proxy with HttpClient in ASP.NET-Core is actually quite simple.
All you need to do is set the handler in the HttpClient-constructor.
Then set the proxy-property of the handler for each request.
Like this:

public class Program
{

    public static async System.Threading.Tasks.Task Main(string[] args)
    {
        string url = "http://aspnetmonsters.com";
        System.Net.WebProxy[] proxies = new[] {
            null,
            new System.Net.WebProxy("104.238.172.20", 8080),
            new System.Net.WebProxy("104.238.167.193", 8080),
            new System.Net.WebProxy("136.244.102.38", 8080),
            new System.Net.WebProxy("95.179.202.40", 8080)
        };

        System.Random rnd = new System.Random();
        using (System.Net.Http.HttpClientHandler handler = new System.Net.Http.HttpClientHandler()
        {
            Proxy = new System.Net.WebProxy("http://127.0.0.1:8888"),
            UseProxy = true,
        })
        {

            using (System.Net.Http.HttpClient hc = new System.Net.Http.HttpClient(handler))
            {
                System.Console.WriteLine("Starting connections");
                for (int i = 0; i < 10; i++)
                {
                    handler.Proxy = proxies[rnd.Next(proxies.Length)];
                    await hc.GetAsync(url);

                    // https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent
                    hc.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (iPad; CPU OS 12_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148");
                    hc.DefaultRequestHeaders.Add("Accept-Language", "fr-FR, fr;q=0.9, en;q=0.8, it;q=0.7, *;q=0.5");
                    hc.DefaultRequestHeaders.Add("Referer", "https://www.baidu.com");

                    using (System.Net.Http.HttpResponseMessage response = await hc.GetAsync(url))
                    {
                        // using (var fs = new System.IO.MemoryStream())
                        // { await response.Content.CopyToAsync(fs); }
                        byte[] ba = await response.Content.ReadAsByteArrayAsync();

                    } // End Using response 

                } // Next i 

                System.Console.WriteLine("Ending connections");
            } // End Using hc 

        } // End Using handler 

        System.Console.WriteLine("--- Press any key to continue --- ");
        System.Console.ReadKey();
    } // End Task Main 

} // End Class Program