DeflatorInputStream和DeflatorOutputStream无法重构原始数据

Question

我想压缩一些数据，因此我了解到DeflatorInputStream和DeflatorOutputStream类。然而，以下例子显示当我使用这些类时，似乎无法重构我的原始数据。

当我切换到ZipInputStream和ZipOutputStream时，可以工作，但由于我不需要zip文件本身，所以认为通用压缩会更好。主要是我有兴趣了解为什么这个例子不能工作。

//Create some "random" data
int bytesLength = 1024;
byte[] bytes = new byte[bytesLength];
for(int i = 0; i < bytesLength; i++) {
     bytes[i] = (byte) (i % 10);
}

//Compress the data, and write it to somewhere (a byte array for this example)
ByteArrayOutputStream arrayOutputStream = new ByteArrayOutputStream();
DeflaterOutputStream outputStream = new DeflaterOutputStream(arrayOutputStream);
outputStream.write(bytes);

//Read and decompress the data
byte[] readBuffer = new byte[5000];
ByteArrayInputStream arrayInputStream = new ByteArrayInputStream(arrayOutputStream.toByteArray());
DeflaterInputStream inputStream = new DeflaterInputStream(arrayInputStream);
int read = inputStream.read(readBuffer);

//Should hold the original (reconstructed) data
byte[] actuallyRead = Arrays.copyOf(readBuffer, read);

//Results differ - will print false
System.out.println(Arrays.equals(bytes, actuallyRead));

- DieterDP

Answer 1

这要怪历史先例。在Unix上，用于反转deflate的函数被称为inflate。因此，与许多其他Java IO类不同，输入和输出流对没有（明显的）匹配名称。

DeflaterOutputStream实际上不允许您反转压缩，而是将字节从汇聚处传递到源端时进行压缩。 DeflaterInputStream也执行压缩操作，但它会在数据从源到汇聚处流动时执行。

为了以未压缩（解压后）格式读取数据，您需要使用InflaterInputStream：

InflaterInputStream inputStream = new InflaterInputStream(arrayInputStream);

另外，由于在一个read调用中可能无法获取流中的所有压缩数据，因此您需要使用循环。可以像这样：

int read;
byte[] finalBuf = new byte[0], swapBuf;
byte[] readBuffer = new byte[5012];

ByteArrayInputStream arrayInputStream = new ByteArrayInputStream(
        compressed);
InflaterInputStream inputStream = new InflaterInputStream(
        arrayInputStream);
while ((read = inputStream.read(readBuffer)) != -1) {
    System.out.println("Intermediate read: " + read);
    swapBuf = finalBuf;
    finalBuf = new byte[swapBuf.length + read];
    System.arraycopy(swapBuf, 0, finalBuf, 0, swapBuf.length);
    System.arraycopy(readBuffer, 0, finalBuf, swapBuf.length, read);
}

最后，确保在检索压缩字节之前刷新您的压缩器输出流（或者关闭该流）。

- Perception

Answer 2

只需要进行两个小改动，就可以让你的代码正常工作。

//Compress the data, and write it to somewhere (a byte array for this example)
ByteArrayOutputStream arrayOutputStream = new ByteArrayOutputStream();
DeflaterOutputStream outputStream = new DeflaterOutputStream(arrayOutputStream);
outputStream.write(bytes);
outputStream.close();

首先，您需要 close() 输出流。压缩器必须执行一些最终步骤以完成其工作。

InflaterInputStream inputStream = new InflaterInputStream(arrayInputStream);

如果您使用一个Deflator InputStream，您将会再次压缩已经被压缩的数据。请替换为Inflator InputStream，您的代码将正常工作。

- AleX

Answer 3

好的，我添加这个回复是因为我在使用Java和SpringBoot（RestTemplate）消费IBM DataStage v11.7的API时遇到了非常痛苦的实现。我不确定是否有其他人可能会遇到类似产品或其他问题...但这些天我遇到了PKIX、使用自签名证书公开的API以及最后一个deflate头的组合问题。很抱歉要用三个回复来做一个大帖子，但你可以找到类似的解决方案来解决前两个问题（PKIX或使用自签名证书和RestTemplate对象的端点）。我在这里添加这个回复是因为Inflater还没有很多方法。

其他一些答案说使用自定义httpclient与resttemplate可能有所帮助，或者可能已经包括自动“解压缩”，但不知何故，这种情况并不相同。

解压或膨胀响应有点棘手，而在末尾添加一个额外的字节则是“樱桃上的奶油”。我设法在挣扎了一会儿后得到了响应，但它打印出了奇怪/特殊字符。我以为这是某种编码问题，但实际上是Inflater的问题。我不得不进行调试以确定响应大小（长度）实际上与我在Postman中收到的相同。所以我说，好吧，长度相似，那么可能发生了什么。因此，感谢所有其他为我们的解决方案做出贡献的开发人员。

import java.security.KeyManagementException;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.X509Certificate;
import java.util.HashMap;
import java.util.Map;
import java.util.Map.Entry;

import javax.net.ssl.SSLContext;

import org.apache.http.client.HttpClient;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.ssl.TrustStrategy;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.ResponseEntity;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.web.client.RestTemplate;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.zip.DataFormatException;
import java.util.zip.Inflater;
import java.util.zip.InflaterInputStream;

    private static Map<String, String> headersMap = new HashMap<String, String>() {
        {
            put("Authorization", "Basic XYZ");
            put("Accept", "*/*");
            put("Accept-Encoding", "deflate");
            put("Connection", "keep-alive");
        }
    };

    private static final String baseUrlDesigner = "https://{HOST}/ibm/iis/api/dscdesignerapi";

    private static final String designerGetAllJobs = "?api=getJobList&hostName={HOST}&projectName={PROJECT}&compress=nowrap9";

    public static RestTemplate getRestTemplateNoSSLValidation( ) {
        TrustStrategy acceptingTrustStrategy = (X509Certificate[] chain, String authType) -> true;
        SSLContext sslContext = null;
        try {
            sslContext = org.apache.http.ssl.SSLContexts.custom()
                    .loadTrustMaterial(null, acceptingTrustStrategy)
                    .build();
        } catch (KeyManagementException e2) {
            e2.printStackTrace();
        } catch (NoSuchAlgorithmException e2) {
            e2.printStackTrace();
        } catch (KeyStoreException e2) {
            e2.printStackTrace();
        }

        SSLConnectionSocketFactory csf = new SSLConnectionSocketFactory(sslContext);

        HttpClient httpClient = HttpClients.custom()
                .setSSLSocketFactory(csf)               
                .disableContentCompression()
                .build();

        HttpComponentsClientHttpRequestFactory requestFactory =
                new HttpComponentsClientHttpRequestFactory();

        requestFactory.setHttpClient(httpClient);

        RestTemplate restTemplate = new RestTemplate(requestFactory);
        return restTemplate;
    }

    public static <T> ResponseEntity<T> sendRequest(String url, HttpMethod method, Map<String,String> headerList, Class<T> responseType) {    
        RestTemplate restTemplate = getRestTemplateNoSSLValidation(); 
        HttpHeaders headers = new HttpHeaders();        
        for (Entry<String,String> e:headerList.entrySet())          
            headers.add(e.getKey(),e.getValue());
        
        HttpEntity<String> request = new HttpEntity<String>(null, headers);     

        ResponseEntity<T> result = (ResponseEntity<T>) restTemplate.exchange(url, method, request, responseType);       
        
        return result;      
    }       

    public static String decompress(byte[] bytes) {
        Inflater inf = new Inflater(true);
        InputStream in = new InflaterInputStream(new ByteArrayInputStream(bytes), inf);
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        try {
            byte[] buffer = new byte[8192];
            int len;
            while ((len = in.read(buffer)) > 0)
                baos.write(buffer, 0, len);
            baos.write(new byte[1], 0, 1); // FIX. Had to force one extra byte, otherwise the body string was not read properly :shrug:
            return new String(baos.toByteArray(), "UTF-8");
        } catch (IOException e) {
            throw new AssertionError(e);
        }
    }


    public static void main(String[] args) throws IOException {
        ResponseEntity<byte[]> responseBytes =
          sendRequest(baseUrlDesigner + designerGetAllJobs,
                HttpMethod.GET, headersMap, byte[].class);
        String respString = decompress(responseBytes.getBody());
        System.out.println("Response lenght: " + respString.length());
        System.out.println("Response body: " + respString);
    }

- gaps96