Jsoup的HTTP日志记录

Question

有没有一种方法记录HTTP请求和响应？假设以下是请求：

Connection.Response res = Jsoup.connect("LOGIN_URL_HERE")
            .data("user", "USER", "pass", "PASS")
            .method(Connection.Method.POST)
            .execute();

我该如何记录Http请求和响应？请注意，我需要的是HTTP而不仅仅是将被解析的HTML。

- Alkis Kalogeris

---

请点击此处查看有关如何为Java HttpURLConnection启用Wire Logging的内容。 - t _ liang

Answer 1

默认情况下，jsoup使用java.net.HttpURLConnection的实现。因此，我认为您需要为该实现开启日志记录（可能是：sun.net.www.protocol.http.HttpURLConnection) 或 java.net）。

有一个系统属性可以启用java net utils的日志记录。

-Djavax.net.debug=all

- Liviu Stirb

Answer 2

由于Jsoup缺少日志记录（我使用的版本为1.12.1），并且使用-Djavax.net.debug=all JVM参数记录太冗长，我找到的最好方法是装饰HttpConnection类，以便可以自定义记录内容。为此，必须围绕Connection.Request和Connection.Response的属性记录execute方法调用。

使用SLF4J的示例实现：

import org.jsoup.Connection;
import org.jsoup.helper.HttpConnection;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.IOException;

public class DiagnosticConnection extends HttpConnection {
    static final Logger LOG = LoggerFactory.getLogger(DiagnosticConnection.class);

    @Override
    public Connection.Response execute() throws IOException {
        log(this.request());
        Connection.Response response = super.execute();
        log(response);

        return response;
    }

    public static Connection connect(String url) {
        Connection connection = new DiagnosticConnection();
        connection.url(url);
        return connection;
    }

    private static void log(Connection.Request request) {
        LOG.info("========================================");
        LOG.info("[url] {}", request.url());
        LOG.info("== REQUEST ==");
        logBase(request);
        LOG.info("[method] {}", request.method());
        LOG.info("[data] {}", request.data());
        LOG.info("[request body] {}", request.requestBody());
    }

    private static void log(Connection.Response response) {
        LOG.info("== RESPONSE ==");
        logBase(response);
        LOG.info("[code] {}", response.statusCode());
        LOG.info("[status msg] {}", response.statusMessage());
        LOG.info("[body] {}", response.body());
        LOG.info("========================================");
    }

    private static void logBase(Connection.Base<?> base) {
        LOG.info("[headers] {}", base.headers());
        LOG.info("[cookies] {}", base.cookies());
    }

}

使用装饰器时，应该使用DiagnosticConnection.connect(<URL>)而不是Jsoup.connect(<URL>)

- Gergely Toth

---

我遇到了错误，因为HttpConnection没有公共构造函数。错误信息：org.jsoup.helper.HttpConnection中没有可用的默认构造函数。 - Osvel Alvarez Jacomino

Answer 3

根据Gergely Toth的回复，我创建了自己的LoggerHttpConnection并正在使用它。

import android.util.Log
import org.jsoup.Connection
import org.jsoup.helper.HttpConnection
import org.jsoup.nodes.Document
import org.jsoup.parser.Parser
import java.io.InputStream
import java.net.Proxy
import java.net.URL
import javax.net.ssl.SSLSocketFactory

class LoggerHttpConnection private constructor(
    private val delegate: HttpConnection,
    private val saveFile: Boolean
) : Connection {

    private val tag = "LoggerHttpConnection"

    companion object {
        fun connect(url: String, saveFile: Boolean = false): LoggerHttpConnection {
            return LoggerHttpConnection(
                HttpConnection.connect(url) as HttpConnection,
                saveFile
            )
        }
    }

    private fun log(request: Connection.Request): String {
        Log.i(tag, "========================================")
        var line = "[url] ${request.url()}"
        var log = "$line\n\n== REQUEST ==\n"
        Log.i(tag, line)

        Log.i(tag, "== REQUEST ==")
        log += logBase(request)

        line = "[method] ${request.method()}"
        log += "$line\n"
        Log.i(tag, line)

        for (data in request.data()) {
            line = "[data] ${data.key()}=${data.value()}"
            log += "$line\n"
            Log.i(tag, line)
        }

        line = "[request body] ${request.requestBody()}"
        log += "$line\n"
        Log.i(tag, line)

        return log
    }

    private fun log(response: Connection.Response): String {
        var line = ""
        var log = "\n== RESPONSE ==\n"

        Log.i(tag, "== RESPONSE ==")
        log += logBase(response)

        line = "[code] ${response.statusCode()}"
        log += "$line\n"
        Log.i(tag, line)

        line = "[status msg] ${response.statusMessage()}"
        log += "$line\n"
        Log.i(tag, line)

        line = "[body] ${response.body()}"
        log += "$line\n"
        Log.i(tag, line)

        Log.i(tag, "========================================")

        return log
    }

    private fun logBase(base: Connection.Base<*>): String {
        var line = ""
        var log = ""
        for (header in base.headers()) {
            line = "[header] ${header.key}=${header.value}"
            log += "$line\n"
            Log.i(tag, line)
        }
        for (cookie in base.cookies()) {
            line = "[cookie] ${cookie.key}: ${cookie.value}"
            log += "$line\n"
            Log.i(tag, line)
        }
        return log
    }

    override fun execute(): Connection.Response {
        var logs = log(request())
        val response = delegate.execute()
        logs += log(response)
        if (saveFile)
            logs.saveToFile("request_log") //do something to save your log in a file if its necesary
        return response
    }

    override fun ignoreContentType(ignoreContentType: Boolean): Connection {
        delegate.ignoreContentType(ignoreContentType)
        return this
    }

    override fun postDataCharset(charset: String?): Connection {
        delegate.postDataCharset(charset)
        return this
    }

    override fun get(): Document {
        return delegate.get()
    }

    override fun post(): Document {
        return delegate.post()
    }

    /** Continue implementing necessary methods for Connection */

}

现在只需使用LoggerHttpConnection而不是Jsoup来声明您的请求，一切都会正常工作。

Connection.Response res = LoggerHttpConnection.connect("LOGIN_URL_HERE")
        .data("user", "USER", "pass", "PASS")
        .method(Connection.Method.POST)
        .execute();

- Osvel Alvarez Jacomino