我知道
我尝试了多种解决方案来手动解压缩响应,但最终都出现了异常,例如
okhttp3
库默认添加头部Accept-Encoding: gzip
并自动为我们解码响应。然而,我遇到了一个只接受Accept-Encoding: gzip, deflate
头部的主机。如果我不加上deflate
部分,则会失败。现在当我手动将该头部添加到okhttp客户端时,该库不再为我执行解压缩操作。我尝试了多种解决方案来手动解压缩响应,但最终都出现了异常,例如
java.util.zip.ZipException: Not in GZIP format
。以下是我迄今为止尝试过的方法://decompresser
public static String decompressGZIP(InputStream inputStream) throws IOException
{
InputStream bodyStream = new GZIPInputStream(inputStream);
ByteArrayOutputStream outStream = new ByteArrayOutputStream();
byte[] buffer = new byte[4096];
int length;
while ((length = bodyStream.read(buffer)) > 0)
{
outStream.write(buffer, 0, length);
}
return new String(outStream.toByteArray());
}
//run scraper
scrape(api, new Callback()
{
// Something went wrong
@Override
public void onFailure(@NonNull Call call, @NonNull IOException e)
{
}
@Override
public void onResponse(@NonNull Call call, @NonNull Response response) throws IOException
{
if (response.isSuccessful())
{
try
{
InputStream responseBodyBytes = responseBody.byteStream();
returnedObject = GZIPCompression.decompress(responseBodyBytes);
if (returnedObject != null)
{
String htmlResponse = returnedObject.toString();
}
}
catch (ProtocolException e){}
if(response != null) response.close();
}
}
});
private Call scrape(Map<?, ?> api, Callback callback)
{
MediaType JSON = MediaType.parse("application/json; charset=utf-8");
String method = (String) api.get("method");
String url = (String) api.get("url");
Request.Builder requestBuilder = new Request.Builder().url(url);
RequestBody requestBody;
requestBuilder.header("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0");
requestBuilder.header("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
requestBuilder.header("Accept-Language", "en-US,en;q=0.5");
requestBuilder.header("Accept-Encoding", "gzip, deflate");
requestBuilder.header("Connection", "keep-alive");
requestBuilder.header("Upgrade-Insecure-Requests", "1");
requestBuilder.header("Cache-Control", "max-age=0");
Request request = requestBuilder.build();
Call call = client.newCall(request);
call.enqueue(callback);
return call;
}
请注意,响应头将始终返回Content-Encoding: gzip
和Transfer-Encoding: chunked
还有一件事,我也尝试过这个主题中的解决方案,但仍然出现D/OkHttp: java.io.IOException: ID1ID2: actual 0x00003c68 != expected 0x00001f8b
。
任何帮助都将不胜感激。