HTMLagilityPack与Powershell结合使用，Windows身份验证

Question

我有一个叫做Lansweeper的工具，它在本地服务器上运行。现在我想从它那里抓取一个页面，但是它使用Windows身份验证。我使用PowerShell作为脚本语言。我主要使用HTMLAgilityPack进行抓取，但我从未抓取过使用Windows身份验证的页面。

有人知道如何通过它传递我的凭据吗？以便在特定凭据下打开页面？（例如我的管理员帐户而不是我的普通帐户）。 （是的，我可以将我的普通用户添加到允许的用户中，但这不是我想使用的解决方案）。

我尝试了以下方法，但它不起作用。

[Reflection.Assembly]::LoadFile("C:\Scraping\HtmlAgilityPack\lib\Net45\HtmlAgilityPack.dll”)
[HtmlAgilityPack.HtmlWeb]$web = @{}
$webclient = new-object System.Net.WebClient
$username = "user"
$password = "passw0rd-"
$domain = "mydomain"
$webclient.Credentials = new-object System.Net.NetworkCredential($username, $password, $domain)
[HtmlAgilityPack.HtmlDocument]$doc = $web.Load("http://lansweeper:81/user.aspx?username=sam&userdomain=mydomain","","",$webclient.Credentials) 
[HtmlAgilityPack.HtmlNodeCollection]$nodes = $doc.DocumentNode.SelectNodes("//body")

我一直在研究这些函数，看到了两种可能性：

TypeName   : HtmlAgilityPack.HtmlWeb
Name       : Load
HtmlAgilityPack.HtmlDocument Load(string url), 
HtmlAgilityPack.HtmlDocument Load(string url, string proxyHost, int proxyPort, string userId, string password), 
HtmlAgilityPack.HtmlDocument Load(string url, string method), 
HtmlAgilityPack.HtmlDocument Load(string url, string method, System.Net.WebProxy proxy, System.Net.NetworkCredential credentials)

Name       : Get
MemberType : Method
void Get(string url, string path), 
void Get(string url, string path, System.Net.WebProxy proxy, System.Net.NetworkCredential credentials), 
void Get(string url, string path, string method), 
void Get(string url, string path, System.Net.WebProxy proxy, System.Net.NetworkCredential credentials, string method)

但我无法让其中之一正常工作。有人曾经用过Powershell做过这个吗？

- Snak3d0c

Answer 1

我找到了如何做到这一点：希望它能帮助未来的某个人。 要想找出答案并不容易，但是一旦你看到了，就很容易了。

[Reflection.Assembly]::LoadFile("C:\temp\HtmlAgilityPack\lib\Net45\HtmlAgilityPack.dll") | Out-Null
[HtmlAgilityPack.HtmlWeb]$web = @{}
$url = "http://lansweeper:81/user.aspx?username=sam&userdomain=mydomain"
$webclient = new-object System.Net.WebClient

    $cred = new-object System.Net.NetworkCredential
    $defaultCredentials =  $cred.UseDefaultCredentials

$proxyAddr = (get-itemproperty 'HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings').ProxyServer
$proxy = new-object System.Net.WebProxy
$proxy.Address = $proxyAddr
$proxy.useDefaultCredentials = $true 
$proxy

[HtmlAgilityPack.HtmlDocument]$doc = $web.Load($url,"GET","$proxy",$defaultCredentials ) 
[HtmlAgilityPack.HtmlNodeCollection]$nodes = $doc.DocumentNode.SelectNodes("//html[1]/body[1]")

$nodes

<# USER RESOURCES
https://msdn.microsoft.com/en-us/library/system.net.webclient.usedefaultcredentials(v=vs.110).aspx
https://forums.asp.net/t/2027997.aspx?HtmlAgilityPack+Stuck+trying+to+understand+HtmlWeb+Load+NetworkCredential
https://msdn.microsoft.com/en-us/library/system.net.webclient.usedefaultcredentials.aspx
https://dev59.com/A3RB5IYBdhLWcg3wn4YL

TypeName   : HtmlAgilityPack.HtmlWeb
Name       : Load
HtmlAgilityPack.HtmlDocument Load(string url, string proxyHost, int proxyPort, string userId, string password), 
HtmlAgilityPack.HtmlDocument Load(string url, string method, System.Net.WebProxy proxy, System.Net.NetworkCredential credentials)
#>

- Snak3d0c