我已经使用Selenium2 / WebDriver编写了测试，想要测试HTTP请求返回的HTTP 403 Forbidden状态。
使用Selenium WebDriver能否获取HTTP响应状态码？
11个回答
85
总的来说，不行。使用Selenium WebDriver API是不可能实现的。这一点在该项目的问题跟踪器中已经被讨论得很透彻了，并且该功能将不会添加到API中。
- JimEvans
1
61是的。但是 API 没有、并且也不会揭示这么简单而关键的信息，这完全是一个笑话。 - 0b101010
65
使用Selenium和Chrome或Firefox，可以获取HTTP请求的响应代码。只需以日志模式启动Chrome或Firefox，并告诉chromedriver执行“Network.enable”即可。以下是java + Selenium + Chrome的示例，但我猜任何语言（Python、C#等）都可以实现。要实现这一点，您需要启用性能日志记录。
LoggingPreferences logPrefs = new LoggingPreferences();
logPrefs.enable(LogType.PERFORMANCE, Level.ALL);
cap.setCapability(CapabilityType.LOGGING_PREFS, logPrefs);
在请求完成后，您需要获取并迭代性能日志，查找所请求的url的"Network.responseReceived"：
LogEntries logs = driver.manage().logs().get("performance");
以下是代码：
import java.util.Iterator;
import java.util.logging.Level;
import org.json.JSONException;
import org.json.JSONObject;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.logging.LogEntries;
import org.openqa.selenium.logging.LogEntry;
import org.openqa.selenium.logging.LogType;
import org.openqa.selenium.logging.LoggingPreferences;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;
public class TestResponseCode
{
public static void main(String[] args)
{
// simple page (without many resources so that the output is
// easy to understand
String url = "http://www.york.ac.uk/teaching/cws/wws/webpage1.html";
DownloadPage(url);
}
private static void DownloadPage(String url)
{
ChromeDriver driver = null;
try
{
ChromeOptions options = new ChromeOptions();
// add whatever extensions you need
// for example I needed one of adding proxy, and one for blocking
// images
// options.addExtensions(new File(file, "proxy.zip"));
// options.addExtensions(new File("extensions",
// "Block-image_v1.1.crx"));
DesiredCapabilities cap = DesiredCapabilities.chrome();
cap.setCapability(ChromeOptions.CAPABILITY, options);
// set performance logger
// this sends Network.enable to chromedriver
LoggingPreferences logPrefs = new LoggingPreferences();
logPrefs.enable(LogType.PERFORMANCE, Level.ALL);
cap.setCapability(CapabilityType.LOGGING_PREFS, logPrefs);
driver = new ChromeDriver(cap);
// navigate to the page
System.out.println("Navigate to " + url);
driver.navigate().to(url);
// and capture the last recorded url (it may be a redirect, or the
// original url)
String currentURL = driver.getCurrentUrl();
// then ask for all the performance logs from this request
// one of them will contain the Network.responseReceived method
// and we shall find the "last recorded url" response
LogEntries logs = driver.manage().logs().get("performance");
int status = -1;
System.out.println("\nList of log entries:\n");
for (Iterator<LogEntry> it = logs.iterator(); it.hasNext();)
{
LogEntry entry = it.next();
try
{
JSONObject json = new JSONObject(entry.getMessage());
System.out.println(json.toString());
JSONObject message = json.getJSONObject("message");
String method = message.getString("method");
if (method != null
&& "Network.responseReceived".equals(method))
{
JSONObject params = message.getJSONObject("params");
JSONObject response = params.getJSONObject("response");
String messageUrl = response.getString("url");
if (currentURL.equals(messageUrl))
{
status = response.getInt("status");
System.out.println(
"---------- bingo !!!!!!!!!!!!!! returned response for "
+ messageUrl + ": " + status);
System.out.println(
"---------- bingo !!!!!!!!!!!!!! headers: "
+ response.get("headers"));
}
}
} catch (JSONException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
}
System.out.println("\nstatus code: " + status);
} finally
{
if (driver != null)
{
driver.quit();
}
}
}
}
输出结果如下：
Navigate to http://www.york.ac.uk/teaching/cws/wws/webpage1.html
List of log entries:
{"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Page.frameAttached","params":{"parentFrameId":"172.1","frameId":"172.2"}}}
{"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Page.frameStartedLoading","params":{"frameId":"172.2"}}}
{"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Page.frameNavigated","params":{"frame":{"securityOrigin":"://","loaderId":"172.1","name":"chromedriver dummy frame","id":"172.2","mimeType":"text/html","parentId":"172.1","url":"about:blank"}}}}
{"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Page.frameStoppedLoading","params":{"frameId":"172.2"}}}
{"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Page.frameStartedLoading","params":{"frameId":"3928.1"}}}
{"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Network.requestWillBeSent","params":{"request":{"headers":{"Upgrade-Insecure-Requests":"1","User-Agent":"Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36"},"initialPriority":"VeryHigh","method":"GET","mixedContentType":"none","url":"http://www.york.ac.uk/teaching/cws/wws/webpage1.html"},"frameId":"3928.1","requestId":"3928.1","documentURL":"http://www.york.ac.uk/teaching/cws/wws/webpage1.html","initiator":{"type":"other"},"loaderId":"3928.1","wallTime":1.47619492749007E9,"type":"Document","timestamp":20226.652971}}}
{"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Network.responseReceived","params":{"frameId":"3928.1","requestId":"3928.1","response":{"headers":{"Accept-Ranges":"bytes","Keep-Alive":"timeout=4, max=100","Cache-Control":"max-age=300","Server":"Apache/2.2.22 (Ubuntu)","Connection":"Keep-Alive","Content-Encoding":"gzip","Vary":"Accept-Encoding","Expires":"Tue, 11 Oct 2016 14:13:47 GMT","Content-Length":"1957","Date":"Tue, 11 Oct 2016 14:08:47 GMT","Content-Type":"text/html"},"connectionReused":false,"timing":{"pushEnd":0,"workerStart":-1,"proxyEnd":-1,"workerReady":-1,"sslEnd":-1,"pushStart":0,"requestTime":20226.65335,"sslStart":-1,"dnsStart":0,"sendEnd":31.6569999995409,"connectEnd":31.4990000006219,"connectStart":0,"sendStart":31.5860000009707,"dnsEnd":0,"receiveHeadersEnd":115.645999998378,"proxyStart":-1},"encodedDataLength":-1,"remotePort":80,"mimeType":"text/html","headersText":"HTTP/1.1 200 OK\r\nDate: Tue, 11 Oct 2016 14:08:47 GMT\r\nServer: Apache/2.2.22 (Ubuntu)\r\nAccept-Ranges: bytes\r\nCache-Control: max-age=300\r\nExpires: Tue, 11 Oct 2016 14:13:47 GMT\r\nVary: Accept-Encoding\r\nContent-Encoding: gzip\r\nContent-Length: 1957\r\nKeep-Alive: timeout=4, max=100\r\nConnection: Keep-Alive\r\nContent-Type: text/html\r\n\r\n","securityState":"neutral","requestHeadersText":"GET /teaching/cws/wws/webpage1.html HTTP/1.1\r\nHost: www.york.ac.uk\r\nConnection: keep-alive\r\nUpgrade-Insecure-Requests: 1\r\nUser-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r\nAccept-Encoding: gzip, deflate, sdch\r\nAccept-Language: en-GB,en-US;q=0.8,en;q=0.6\r\n\r\n","url":"http://www.york.ac.uk/teaching/cws/wws/webpage1.html","protocol":"http/1.1","fromDiskCache":false,"fromServiceWorker":false,"requestHeaders":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","Upgrade-Insecure-Requests":"1","Connection":"keep-alive","User-Agent":"Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36","Host":"www.york.ac.uk","Accept-Encoding":"gzip, deflate, sdch","Accept-Language":"en-GB,en-US;q=0.8,en;q=0.6"},"remoteIPAddress":"144.32.128.84","statusText":"OK","connectionId":11,"status":200},"loaderId":"3928.1","type":"Document","timestamp":20226.770012}}}
---------- bingo !!!!!!!!!!!!!! returned response for http://www.york.ac.uk/teaching/cws/wws/webpage1.html: 200
---------- bingo !!!!!!!!!!!!!! headers: {"Accept-Ranges":"bytes","Keep-Alive":"timeout=4, max=100","Cache-Control":"max-age=300","Server":"Apache/2.2.22 (Ubuntu)","Connection":"Keep-Alive","Content-Encoding":"gzip","Vary":"Accept-Encoding","Expires":"Tue, 11 Oct 2016 14:13:47 GMT","Content-Length":"1957","Date":"Tue, 11 Oct 2016 14:08:47 GMT","Content-Type":"text/html"}
{"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Network.dataReceived","params":{"dataLength":2111,"requestId":"3928.1","encodedDataLength":1460,"timestamp":20226.770425}}}
{"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Page.frameNavigated","params":{"frame":{"securityOrigin":"http://www.york.ac.uk","loaderId":"3928.1","id":"3928.1","mimeType":"text/html","url":"http://www.york.ac.uk/teaching/cws/wws/webpage1.html"}}}}
{"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Network.dataReceived","params":{"dataLength":1943,"requestId":"3928.1","encodedDataLength":825,"timestamp":20226.782673}}}
{"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Network.loadingFinished","params":{"requestId":"3928.1","encodedDataLength":2285,"timestamp":20226.770199}}}
{"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Page.loadEventFired","params":{"timestamp":20226.799391}}}
{"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Page.frameStoppedLoading","params":{"frameId":"3928.1"}}}
{"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Page.domContentEventFired","params":{"timestamp":20226.845769}}}
{"webview":"3b8eaedb-bd0f-4baa-938d-4aee4039abfe","message":{"method":"Network.requestWillBeSent","params":{"request":{"headers":{"Referer":"http://www.york.ac.uk/teaching/cws/wws/webpage1.html","User-Agent":"Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36"},"initialPriority":"High","method":"GET","mixedContentType":"none","url":"http://www.york.ac.uk/favicon.ico"},"frameId":"3928.1","requestId":"3928.2","documentURL":"http://www.york.ac.uk/teaching/cws/wws/webpage1.html","initiator":{"type":"other"},"loaderId":"3928.1","wallTime":1.47619492768527E9,"type":"Other","timestamp":20226.848174}}}
status code: 200
Java + Selenium + Firefox
为了实现Firefox浏览器的操作，在启动Firefox时需要使用MOZ_LOG和MOZ_LOG_FILE环境变量，并将http请求的日志记录级别设置为调试级别(4 = PR_LOG_DEBUG) - map.put("MOZ_LOG", "timestamp,sync,nsHttp:4")。将日志保存在临时文件中，然后获取保存的日志文件内容并使用一些简单的正则表达式来解析响应代码。首先检测请求的开始，识别其ID(nsHttpChannel::BeginConnect [this=000000CED8094000])，然后在第二步中查找该请求ID的响应代码(nsHttpChannel::ProcessResponse [this=000000CED8094000 httpStatus=200])。
import java.io.File;
import java.io.IOException;
import java.net.MalformedURLException;
import java.util.HashMap;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import org.apache.commons.io.FileUtils;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.GeckoDriverService;
public class TestFirefoxResponse
{
public static void main(String[] args)
throws InterruptedException, IOException
{
GeckoDriverService service = null;
// tell firefox to log http requests
// at level 4 = PR_LOG_DEBUG: debug messages, notices
// you could log everything at level 5, but the log file will
// be larger.
// create a temporary log file that will be parsed for
// response code
Map<String, String> map = new HashMap<String, String>();
map.put("MOZ_LOG", "timestamp,sync,nsHttp:4");
File tempFile = File.createTempFile("mozLog", ".txt");
map.put("MOZ_LOG_FILE", tempFile.getAbsolutePath());
GeckoDriverService.Builder builder = new GeckoDriverService.Builder();
service = builder.usingAnyFreePort()
.withEnvironment(map)
.build();
service.start();
WebDriver driver = new FirefoxDriver(service);
// test 200
String url = "https://api.ipify.org/?format=text";
// test 404
// String url = "https://www.advancedwebranking.com/lsdkjflksdjfldksfj";
driver.get(url);
driver.quit();
String logContent = FileUtils.readFileToString(tempFile);
ParseLog(logContent, url);
}
private static void ParseLog(String logContent, String url) throws MalformedURLException
{
// this is how the log looks like when the request starts
// I have to get the id of the request using a regular expression
// and use that id later to get the response
//
// 2017-11-02 14:14:01.170000 UTC - [Main Thread]: D/nsHttp nsHttpChannel::BeginConnect [this=000000BFF27A5000]
// 2017-11-02 14:14:01.170000 UTC - [Main Thread]: D/nsHttp host=api.ipify.org port=-1
// 2017-11-02 14:14:01.170000 UTC - [Main Thread]: D/nsHttp uri=https://api.ipify.org/?format=text
String pattern = "BeginConnect \\[this=(.*?)\\](?:.*?)uri=(.*?)\\s";
Pattern p = Pattern.compile(pattern, Pattern.CASE_INSENSITIVE | Pattern.DOTALL);
Matcher m = p.matcher(logContent);
String urlID = null;
while (m.find())
{
String id = m.group(1);
String uri = m.group(2);
if (uri.equals(url))
{
urlID = id;
break;
}
}
System.out.println("request id = " + urlID);
// this is how the response looks like in the log file
// ProcessResponse [this=000000CED8094000 httpStatus=200]
// I will use another regular espression to get the httpStatus
//
// 2017-11-02 14:45:39.296000 UTC - [Main Thread]: D/nsHttp nsHttpChannel::OnStartRequest [this=000000CED8094000 request=000000CED8014BB0 status=0]
// 2017-11-02 14:45:39.296000 UTC - [Main Thread]: D/nsHttp nsHttpChannel::ProcessResponse [this=000000CED8094000 httpStatus=200]
pattern = "ProcessResponse \\[this=" + urlID + " httpStatus=(.*?)\\]";
p = Pattern.compile(pattern, Pattern.CASE_INSENSITIVE | Pattern.DOTALL);
m = p.matcher(logContent);
if (m.find())
{
String responseCode = m.group(1);
System.out.println("response code found " + responseCode);
}
else
{
System.out.println("response code not found");
}
}
}
输出结果为
请求 ID = 0000007653D67000 找到响应代码200
响应头也可以在日志文件中找到。 如果需要，您也可以获取它们。
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp http response [
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp HTTP/1.1 404 Not Found
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp Accept-Ranges: bytes
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp Cache-control: no-cache="set-cookie"
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp Content-Type: text/html; charset=utf-8
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp Date: Thu, 02 Nov 2017 14:54:36 GMT
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp ETag: "7969-55bc076a61e80"
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp Last-Modified: Tue, 17 Oct 2017 16:17:46 GMT
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp Server: Apache/2.4.23 (Amazon) PHP/5.6.24
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp Set-Cookie: AWSELB=5F256FFA816C8E72E13AE0B12A17A3D540582F804C87C5FEE323AF3C9B638FD6260FF473FF64E44926DD26221AAD2E9727FD739483E7E4C31784C7A495796B416146EE83;PATH=/
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp Content-Length: 31081
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp Connection: keep-alive
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp OriginalHeaders
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp Accept-Ranges: bytes
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp Cache-control: no-cache="set-cookie"
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp Content-Type: text/html; charset=utf-8
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp Date: Thu, 02 Nov 2017 14:54:36 GMT
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp ETag: "7969-55bc076a61e80"
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp Last-Modified: Tue, 17 Oct 2017 16:17:46 GMT
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp Server: Apache/2.4.23 (Amazon) PHP/5.6.24
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp Set-Cookie: AWSELB=5F256FFA816C8E72E13AE0B12A17A3D540582F804C87C5FEE323AF3C9B638FD6260FF473FF64E44926DD26221AAD2E9727FD739483E7E4C31784C7A495796B416146EE83;PATH=/
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp Content-Length: 31081
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp Connection: keep-alive
2017-11-02 14:54:36.775000 UTC - [Socket Thread]: I/nsHttp ]
2017-11-02 14:54:36.775000 UTC - [Main Thread]: D/nsHttp nsHttpChannel::OnStartRequest [this=0000008A65D85000 request=0000008A65D1F900 status=0]
2017-11-02 14:54:36.775000 UTC - [Main Thread]: D/nsHttp nsHttpChannel::ProcessResponse [this=0000008A65D85000 httpStatus=404]
- Stefan Matei
11
3对于查看此内容的人，请注意，chromium的network.enable在启用时会向每个HTTP请求中插入一个额外的标头。 - Fake Name
2嗨，我能得到响应代码，但是如何获取响应正文？我在响应JSONObject中没有看到它。 - LINGS
1这似乎在Selenium 4.0.0-alpha-4中无法工作。包括的唯一类型日志是“客户端”。我是否遗漏了什么，或者这个功能在4.0.0-alpha-4中仍在开发中？ - Ivaylo Slavov
4Chrome脚本在我的4.0.0-alpha-4版本中可用，请查看此示例https://github.com/smatei/SeleniumChromeHTTPResponse。 - Stefan Matei
1我认为使用Selenium 4，您可以使用DevTools访问响应。 - tarekahf
显示剩余6条评论
31
如果你使用Python，可以考虑使用Selenium Wire，这是一个用于检查测试期间浏览器发出请求的库。
通过driver.requests属性，你可以访问到请求：
from seleniumwire import webdriver # Import from seleniumwire
# Create a new instance of the Firefox driver
driver = webdriver.Firefox()
# Go to the Google home page
driver.get('https://www.google.com')
# Access requests via the `requests` attribute
for request in driver.requests:
if request.response:
print(
request.url,
request.response.status_code,
request.response.headers['Content-Type']
)
输出：
https://www.google.com/ 200 text/html; charset=UTF-8
https://www.google.com/images/branding/googlelogo/2x/googlelogo_color_120x44dp.png 200 image/png
https://consent.google.com/status?continue=https://www.google.com&pc=s×tamp=1531511954&gl=GB 204 text/html; charset=utf-8
https://www.google.com/images/branding/googlelogo/2x/googlelogo_color_272x92dp.png 200 image/png
https://ssl.gstatic.com/gb/images/i2_2ec824b0.png 200 image/png
https://www.google.com/gen_204?s=webaft&t=aft&atyp=csi&ei=kgRJW7DBONKTlwTK77wQ&rt=wsrt.366,aft.58,prt.58 204 text/html; charset=UTF-8
...
这个库让你能够访问头信息，状态码，正文内容，以及修改头信息和重写URL的能力。
- Will Keeling
10
1我正在尝试使用webdriver.Remote，但是我收到了“AttributeError：'WebDriver'对象没有属性'requests'”的错误消息。这是因为selenium-wire不支持远程驱动程序吗？官方文档非常缺乏。谢谢！ - Mr. Developerdude
1@MohamedImran 这个方法需要一个正则表达式作为参数，所以你可以只传递唯一匹配的URL部分。例如，要匹配
http://myserver.com/some/path/12345/，你可以传递 driver.wait_for_request('.*/12345/')。 - Will Keeling1@MohamedImran 你不需要使用for循环。 - Will Keeling
@SriramIlango 目前 Selenium Wire 只存在于 Python 中。 - Will Keeling
显示剩余5条评论
18
您可以使用BrowserMob代理程序通过HttpRequestInterceptor来捕获请求和响应。以下是Java示例：
// Start the BrowserMob proxy
ProxyServer server = new ProxyServer(9978);
server.start();
server.addResponseInterceptor(new HttpResponseInterceptor()
{
@Override
public void process(HttpResponse response, HttpContext context)
throws HttpException, IOException
{
System.out.println(response.getStatusLine());
}
});
// Get selenium proxy
Proxy proxy = server.seleniumProxy();
// Configure desired capability for using proxy server with WebDriver
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.PROXY, proxy);
// Set up driver
WebDriver driver = new FirefoxDriver(capabilities);
driver.get("https://dev59.com/n2w15IYBdhLWcg3wmc4J");
// Close the browser
driver.quit();
- Christian
2
我正在使用你的解决方案，但是在代理的情况下，页面加载非常缓慢。有什么想法是怎么回事吗？另请参阅我的问题：http://stackoverflow.com/questions/21043928/slow-selenium-tests-when-using-proxy-with-browsermob - Charlie
1原始的BrowserMob代理现在已经不受支持并且正在腐烂。我们有一个开源的分支，其中还内置了性能、页面和网络断言。https://github.com/browserup/browserup-proxy - ebeland
15
我也遇到了同样的问题，卡了几天，但经过一些研究，我发现我们可以实际上使用 Chrome 的 "--remote-debugging-port" 与 Selenium Web Driver 一起拦截请求。
使用以下伪代码作为参考：
使用远程调试创建 Chrome 驱动程序的实例
int freePort = findFreePort();
chromeOptions.addArguments("--remote-debugging-port=" + freePort);
ChromeDriver driver = new ChromeDriver(chromeOptions);`
向http://127.0.0.1:freePort发送GET请求
String response = makeGetCall( "http://127.0.0.1" + freePort + "/json" );
String webSocketUrl = response.substring(response.indexOf("ws://127.0.0.1"), response.length() - 4);
连接到此套接字，您可以使用asyncHttp
socket = maketSocketConnection( webSocketUrl );
启用网络抓包
socket.send( { "id" : 1, "method" : "Network.enable" } );
现在Chrome将发送所有与网络相关的事件并按如下方式捕获它们
socket.onMessageReceived( String message ){
Json responseJson = toJson(message);
if( responseJson.method == "Network.responseReceived" ){
//extract status code
}
}
driver.get("http://stackoverflow.com");
你可以在开发工具网站提到的所有事情中完成。 请参阅https://chromedevtools.github.io/devtools-protocol/ 注意：请使用ChromeDriver 2.39或更高版本。
我希望它能帮助某些人。
- Mayank Senani
4
太棒了，我很高兴有一种方法可以做到这一点！只是我希望有一个库（对于我的情况是Ruby的
selenium-webdriver），可以自动启用 Network.enable 并公开一种简单的方式来检查最后一个响应...比如 driver.response.status_code。也许有一天我会着手添加它... :) - Tyler Rick如果其他Ruby程序员遇到这个问题，我创建了一个gem，https://github.com/TylerRick/capybara-chrome_response_headers，它可以让你通过简单调用`status_code`从Capybara/RSpec测试中轻松获取HTTP状态码。它使用Chrome远程调试协议，就像这个例子一样。 - Tyler Rick
谢谢你，伙计！对我有用。
Chrome 76 + Selenium 3.141 - jessez
你能用PowerShell做同样的事情吗？你能给一些提示吗？我在这里创建了一个新问题：https://stackoverflow.com/questions/71021392/using-selenium-webdriver-pester-and-powershell-to-get-and-verify-the-network-r - tarekahf
15
使用JavaScript在任何语言中获取响应代码：
如果你的Selenium测试在现代浏览器中运行，一种简单的方法是发送同步XMLHttpRequest*并检查响应的status来获取响应代码：
var xhr = new XMLHttpRequest();
xhr.open('GET', 'http://exampleurl.ex', false);
xhr.send(null);
assert(200, xhr.status);
您可以通过请求Selenium执行脚本来使用此技术，适用于任何编程语言。例如，在Java中，您可以使用JavascriptExecutor.executeScript()发送XMLHttpRequest：
final String GET_RESPONSE_CODE_SCRIPT =
"var xhr = new XMLHttpRequest();" +
"xhr.open('GET', arguments[0], false);" +
"xhr.send(null);" +
"return xhr.status";
JavascriptExecutor javascriptExecutor = (JavascriptExecutor) webDriver;
Assert.assertEquals(200,
javascriptExecutor.executeScript(GET_RESPONSE_CODE_SCRIPT, "http://exampleurl.ex"));
* 您可以发送异步的 XMLHttpRequest，但是您需要等待它完成后才能继续测试。
获取Java中的响应代码：
您可以使用URL.openConnection()和HttpURLConnection.getResponseCode()在Java中获取响应代码：
URL url = new URL("http://exampleurl.ex");
HttpURLConnection httpURLConnection = (HttpURLConnection) url.openConnection();
httpURLConnection.setRequestMethod("GET");
// You may need to copy over the cookies that Selenium has in order
// to imitate the Selenium user (for example if you are testing a
// website that requires a sign-in).
Set<Cookie> cookies = webDriver.manage().getCookies();
String cookieString = "";
for (Cookie cookie : cookies) {
cookieString += cookie.getName() + "=" + cookie.getValue() + ";";
}
httpURLConnection.addRequestProperty("Cookie", cookieString);
Assert.assertEquals(200, httpURLConnection.getResponseCode());
这种方法可能也可以推广到其他语言，但需要根据该语言（或库）的API进行修改。
- stiemannkj1
1
非常感谢，你让我的一天变得美好了！JavaScript 部分对我很有帮助。Java 部分在大多数情况下也可能有效，但在我的情况下，我正在使用 Selenium/Selenide 与远程浏览器（在 Docker 容器中），需要在浏览器中执行代码，但 Java 代码将在本地执行（至少我认为是这样）。 - Alan Z.
8
不确定这是否符合您的要求，但我有一个稍微不同的目标，那就是检查远程图像是否存在，以便避免出现403错误，因此您可以使用以下类似的代码：
public static boolean linkExists(String URLName){
try {
HttpURLConnection.setFollowRedirects(false);
HttpURLConnection con = (HttpURLConnection) new URL(URLName).openConnection();
con.setRequestMethod("HEAD");
return (con.getResponseCode() == HttpURLConnection.HTTP_OK);
}
catch (Exception e) {
e.printStackTrace();
return false;
}
}
- Volodymyr Prysiazhniuk
2
4适用于简单情况，但使用此方法不会保存任何浏览器状态（例如用户登录信息）。 - Roger Keays
我相信您可以将凭据作为标头请求的一部分发送，如果您能尝试一下，我会非常感激。 - Volodymyr Prysiazhniuk
3
使用Selenium WebDriver直接无法获取HTTP响应代码。但可以通过Java编写代码获取，并在Selenium WebDriver中使用。
要通过Java获得HTTP响应代码:
public static int getResponseCode(String urlString) throws MalformedURLException, IOException{
URL url = new URL(urlString);
HttpURLConnection huc = (HttpURLConnection)url.openConnection();
huc.setRequestMethod("GET");
huc.connect();
return huc.getResponseCode();
}
现在，您可以按照以下方式编写Selenium WebDriver代码：
private static int statusCode;
public static void main(String... args) throws IOException{
WebDriver driver = new FirefoxDriver();
driver.manage().window().maximize();
driver.get("https://www.google.com/");
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
List<WebElement> links = driver.findElements(By.tagName("a"));
for(int i = 0; i < links.size(); i++){
if(!(links.get(i).getAttribute("href") == null) && !(links.get(i).getAttribute("href").equals(""))){
if(links.get(i).getAttribute("href").contains("http")){
statusCode= getResponseCode(links.get(i).getAttribute("href").trim());
if(statusCode == 403){
System.out.println("HTTP 403 Forbidden # " + i + " " + links.get(i).getAttribute("href"));
}
}
}
}
}
- Ripon Al Wasim
2
1如果这是一个POST表单提交会怎样？如果存在Cookies会怎样？ - nafg
2这只是在Selenium之外发送HTTP请求。 - Matthias Winkelmann
2
您可以尝试使用Mobilenium（https://github.com/rafpyprog/Mobilenium），这是一个将BrowserMob代理和Selenium绑定在一起的Python软件包。
以下是一个使用示例：
以下是一个使用示例：
>>> from mobilenium import mobidriver
>>>
>>> browsermob_path = 'path/to/browsermob-proxy'
>>> mob = mobidriver.Firefox(browsermob_binary=browsermob_path)
>>> mob.get('http://python-requests.org')
301
>>> mob.response['redirectURL']
'http://docs.python-requests.org'
>>> mob.headers['Content-Type']
'application/json; charset=utf8'
>>> mob.title
'Requests: HTTP for Humans \u2014 Requests 2.13.0 documentation'
>>> mob.find_elements_by_tag_name('strong')[1].text
'Behold, the power of Requests'
- Rafael Ribeiro
0
简单的一行代码返回GET请求的响应状态和文本。
response = driver.execute_async_script("var callback = arguments[arguments.length - 1]; fetch('" + url + "', {method: 'GET', headers: " + str(headers) + "}).then((response) => response.text().then((text) => callback({'status': response.status, 'text': text})))")
# status = response['status']
# text = response['text']
- Tom Davenport
网页内容由stack overflow 提供, 点击上面的可以查看英文原文，
原文链接
原文链接
- 相关问题
- 9 如何使用Selenium Python绑定的WebDriver提交HTTP身份验证
- 10 如何从Selenium Webdriver获取异步JavaScript响应
- 3 如何使用Selenium WebDriver获取网页标题？
- 4 如何使用Selenium Webdriver获取所有列表项？
- 10 在Selenium WebDriver（浏览器）中呈现HTTP响应（HTML内容）
- 11 如何使用Selenium WebDriver获取检查元素代码
- 8 通过Selenium WebDriver C#获取HTTP状态码
- 5 有没有办法使用Selenium Webdriver（Firefox）记录HTTP请求/响应？
- 36 如何使用Python和Selenium WebDriver获取localStorage
- 10 如何从Selenium WebDriver Firefox的`driver.page_source`中获取HTTP请求的原始JSON响应
webdriver的限制，这些限制对Python和Java都是相同的;) - maxkoryukov