使用C# WebClient登录accounts.google.com

Question

我尝试使用WebClient来认证accounts.google.com，但是遇到了很大的困难。

我正在使用C# WebClient对象来实现以下功能。

我正在向https://accounts.google.com/ServiceLoginAuth?service=oz提交表单字段。

这里是POST字段：

service=oz
dsh=-8355435623354577691
GALX=33xq1Ma_CKI
timeStmp=
secTok=
Email=[e-mail]
Passwd=password
signIn=Sign in
PersistentCookie=yes
rmShown=1

现在，当登录页面加载时，在我提交数据之前，它具有以下标题：

Content-Type                text/html; charset=UTF-8
Strict-Transport-Security   max-age=2592000; includeSubDomains
Set-Cookie                  GAPS=1:QClFh_dKle5DhcdGwmU3m6FiPqPoqw:SqdLB2u4P2oGjt_x;Path=/;Expires=Sat, 21-Dec-2013 07:31:40 GMT;Secure;HttpOnly
Cache-Control               no-cache, no-store
Pragma                      no-cache
Expires                     Mon, 01-Jan-1990 00:00:00 GMT
X-Frame-Options             Deny
X-Auto-Login                realm=com.google&args=service%3Doz%26continue%3Dhttps%253A%252F%252Faccounts.google.com%252FManageAccount
Content-Encoding            gzip
Transfer-Encoding           chunked
Date                        Thu, 22 Dec 2011 07:31:40 GMT
X-Content-Type-Options      nosniff
X-XSS-Protection            1; mode=block
Server                      GSE

好的，现在我该如何使用WebClient类来包含这些头信息呢？

我尝试过webClient_.Headers.Add();，但它的效果有限，总是返回登录页面。

下面是我使用的一个类。非常感谢任何帮助。

---

获取登录页面

    public void LoginPageRequest(Account acc)
    {

        var rparams = new RequestParams();
        rparams.URL = @"https://accounts.google.com/ServiceLoginAuth?service=oz";
        rparams.RequestName = "LoginPage";
        rparams.Account = acc;

        webClient_.DownloadDataAsync(new Uri(rparams.URL), rparams);
    }

    void webClient__DownloadDataCompleted(object sender, DownloadDataCompletedEventArgs e)
    {
        RequestParams rparams = (RequestParams)e.UserState;

        if (rparams.RequestName == "LoginPage")
        {
            ParseLoginRequest(e.Result, e.UserState);
        }
    }

现在使用HtmlAgilityPack获取表单字段并将它们添加到参数集合中。

    public void ParseLoginRequest(byte[] data, object UserState)
    {
        RequestParams rparams = (RequestParams)UserState;

        rparams.ClearParams();

        ASCIIEncoding encoder = new ASCIIEncoding();

        string html = encoder.GetString(data);

        HtmlNode.ElementsFlags.Remove("form");

        HtmlDocument doc = new HtmlDocument();
        doc.LoadHtml(html);

        HtmlNode form = doc.GetElementbyId("gaia_loginform");

        rparams.URL = form.GetAttributeValue("action", string.Empty);
        rparams.RequestName = "LoginPost";

        var inputs = form.Descendants("input");
        foreach (var element in inputs)
        {
            string name = element.GetAttributeValue("name", "undefined");
            string value = element.GetAttributeValue("value", "");
            if (!name.Equals("undefined")) {

                if (name.ToLower().Equals("email"))
                {
                    value = rparams.Account.Email;
                }
                else if (name.ToLower().Equals("passwd"))
                {
                    value = rparams.Account.Password;
                }

                rparams.AddParam(name,value);
                Console.WriteLine(name + "-" + value);
            }
        }

        webClient_.UploadValuesAsync(new Uri(rparams.URL),"POST", rparams.GetParams,rparams);

我提交数据后，没有被重定向或显示成功信息，反而跳转到了登录页面。

我做错了什么？

- Tim

Answer 1

经过一番摸索，看起来WebClient类并不是解决此特定问题的最佳方法。

为了实现以下目标，我不得不跳到WebRequest的下一个级别。

在进行WebRequest（HttpWebRequest）并使用HttpWebResponse时，可以设置CookieContainer。

        webRequest_ = (HttpWebRequest)HttpWebRequest.Create(rparams.URL);

        webRequest_.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)";
        CookieContainer cookieJar = new CookieContainer();
        webRequest_.CookieContainer = cookieJar;

        string html = string.Empty;

        try
        {
            using (WebResponse response = webRequest_.GetResponse())
            {
                using (var streamReader = new StreamReader(response.GetResponseStream()))
                {
                    html = streamReader.ReadToEnd();
                    ParseLoginRequest(html, response,cookieJar);
                }
            }
        }
        catch (WebException e)
        {
            using (WebResponse response = e.Response)
            {
                HttpWebResponse httpResponse = (HttpWebResponse)response;
                Console.WriteLine("Error code: {0}", httpResponse.StatusCode);
                using (var streamReader = new StreamReader(response.GetResponseStream()))
                    Console.WriteLine(html = streamReader.ReadToEnd());
            }
        }

然后，在进行帖子发布时，使用相同的Cookie容器，方法如下：

        webRequest_ = (HttpWebRequest)HttpWebRequest.Create(rparams.URL);

        webRequest_.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)";
        webRequest_.Method = "POST";
        webRequest_.ContentType = "application/x-www-form-urlencoded";
        webRequest_.CookieContainer = cookieJar;

        var parameters = new StringBuilder();

        foreach (var key in rparams.Params)
        {
            parameters.AppendFormat("{0}={1}&",HttpUtility.UrlEncode(key.ToString()),
                HttpUtility.UrlEncode(rparams.Params[key.ToString()]));
        }

        parameters.Length -= 1;

        using (var writer = new StreamWriter(webRequest_.GetRequestStream()))
        {
            writer.Write(parameters.ToString());
        }

        string html = string.Empty;

        using (response = webRequest_.GetResponse())
        {
            using (var streamReader = new StreamReader(response.GetResponseStream()))
            {
                html = streamReader.ReadToEnd();

            }
        }

所以这里的代码可以运行，但不适用于生产环境，应该进行优化。 请把它当作一个示例。

- Tim

Answer 2

这是一个快速的示例，写在答案窗格中且未经测试。您可能需要从初始请求中解析出一些值，以便将某些表单值放入formData中。我的很多代码都基于这种类型的过程，除非我们需要爬取Facebook Spokeo类型的网站，在这种情况下，ajax会使我们使用不同的方法。

using System;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.Linq;
using System.Text;

namespace GMailTest
{
    class Program
    {
        private static NameValueCollection formData = new NameValueCollection();
        private static CookieAwareWebClient webClient = new CookieAwareWebClient();

        static void Main(string[] args)
        {
            formData.Clear();
            formData["service"] = "oz";
            formData["dsh"] = "-8355435623354577691";
            formData["GALX"] = "33xq1Ma_CKI";
            formData["timeStmp"] = "";
            formData["secTok"] = "";
            formData["Email"] = "[e-mail]";
            formData["Passwd"] = "password";
            formData["signIn"] = "Sign in";
            formData["PersistentCookie"] = "yes";
            formData["rmShown"] = "1";

            byte[] responseBytes = webClient.UploadValues("https://accounts.google.com/ServiceLoginAuth?service=oz", "POST", formData);
            string responseHTML = Encoding.UTF8.GetString(responseBytes);
        }
    }

    public class CookieAwareWebClient : WebClient
    {
        public CookieAwareWebClient() : this(new CookieContainer())
        { }

        public CookieAwareWebClient(CookieContainer c)
        {
            this.CookieContainer = c;
            this.Headers.Add("User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.52 Safari/536.5");
        }

        public CookieContainer CookieContainer { get; set; }

        protected override WebRequest GetWebRequest(Uri address)
        {
            WebRequest request = base.GetWebRequest(address);
            if (request is HttpWebRequest)
            {
                (request as HttpWebRequest).CookieContainer = this.CookieContainer;
            }
            return request;
        }
    }
}

- Rob B

---

看起来你 Rob 从事的是与我相同的行业，很想与你取得联系，讨论一些你在这篇帖子中提到的 AJAX 方法。 - Tim

你能解释一下这些字段formData["dsh"]，formData["GALX"]应该填什么吗？ - Kemal AL GAZZAH