Android: 在使用gson.toJson()时，出现java.lang.OutOfMemoryError: Failed to allocate with free bytes and 70MB until OOM错误。

Question

在我的Android应用程序中，当我尝试将数据同步到服务器时，如果数据大小超过20MB，我会遇到以下异常。我认为这个异常是由于数据量太大造成的。我使用base64编码将位图图像保存为字符串，并缩小了图像大小，这导致了如此巨大的数据。

04-18 13:51:51.957  16199-16816/com.example.myproject.app E/art﹕ Throwing OutOfMemoryError "Failed to allocate a 128887990 byte allocation with 16777216 free bytes and 70MB until OOM"
04-18 13:51:52.037  16199-16816/com.example.myproject.app E/AndroidRuntime﹕ FATAL EXCEPTION: Thread-4482
Process: com.example.myproject.app, PID: 16199
java.lang.OutOfMemoryError: Failed to allocate a 128887990 byte allocation with 16777216 free bytes and 70MB until OOM
    at java.lang.AbstractStringBuilder.enlargeBuffer(AbstractStringBuilder.java:95)
    at java.lang.AbstractStringBuilder.append0(AbstractStringBuilder.java:146)
    at java.lang.StringBuffer.append(StringBuffer.java:219)
    at java.io.StringWriter.write(StringWriter.java:167)
    at com.google.gson.stream.JsonWriter.string(JsonWriter.java:570)
    at com.google.gson.stream.JsonWriter.value(JsonWriter.java:419)
    at com.google.gson.internal.bind.TypeAdapters$16.write(TypeAdapters.java:426)
    at com.google.gson.internal.bind.TypeAdapters$16.write(TypeAdapters.java:410)
    at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
    at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:112)
    at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:239)
    at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
    at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:97)
    at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:61)
    at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)

如何解决这个问题？我知道这是在使用Gson将数据从类转换为json时出现的。以下是我的代码：

SimpleDateFormat dtf=new SimpleDateFormat("yyyy-MM-dd HH:mm:ss",Locale.ENGLISH);
GsonBuilder builder = new GsonBuilder();
builder.registerTypeAdapter(Date.class, new JsonDeserializer<Date>() {

        @Override
        public Date deserialize(JsonElement json, Type type, JsonDeserializationContext deserializationContext) throws JsonParseException {
            String frStr = json.getAsJsonPrimitive().getAsString();
            Date retDate =null;
            try {
                retDate = dtf.parse(frStr);
            } catch (ParseException e) {
                e.printStackTrace();
            }
            return retDate;
        }
    });
    builder.registerTypeAdapter(Date.class, new JsonSerializer<Date>() {
            @Override
            public JsonElement serialize(Date src, Type typeOfSrc, JsonSerializationContext context) {
                String jsDate = dtf.format(src);
                return new JsonPrimitive(jsDate);
            }
    });
    builder.registerTypeAdapter(byte[].class, new JsonDeserializer<byte[]>() {
            @Override
            public byte[] deserialize(JsonElement json, Type type, JsonDeserializationContext deserializationContext) throws JsonParseException {
                return Base64.decode(json.getAsString(), Base64.NO_WRAP);
            }
    });
    gson = builder.create();


    attDataAcc.setAttList(attList);
    String jsonAttAccts = gson.toJson(attDataAcc, AttachmentDataList.class);
        HttpEntity<String> entityAtt = new HttpEntity<String>(jsonAttAccts,headers);
        ResponseEntity<String> restResA = restTemplate.exchange(strUrl+"/saveAttToServer", HttpMethod.POST, entityAtt, String.class);

public class Attachment implements Serializable {

            @DatabaseField(columnName = "id",id = true)
            private String id;

            @DatabaseField(columnName = "user_id")
            private Integer userId;

            @DatabaseField(columnName = "attachment_id")
            private String attachmentId;

            @DatabaseField(columnName = "file_name")
            private String fileName;

            @DatabaseField(columnName = "file_data")
            private String fileData;

            @DatabaseField(columnName = "date",dataType=DataType.DATE)
            private Date date;

            public Attachment() {
                super();
                // TODO Auto-generated constructor stub
            }

            public Attachment(String id, Integer userId, String attachmentId, String fileName, String fileData, Date date) {
                this.id = id;
                this.userId = userId;
                this.attachmentId = attachmentId;
                this.fileName = fileName;
                this.fileData = fileData;
                this.date = date;
            }

            public String getId() {
                return id;
            }

            public void setId(String id) {
                this.id = id;
            }

            public Integer getUserId() {
                return userId;
            }

            public void setUserId(Integer userId) {
                this.userId = userId;
            }

            public String getAttachmentId() {
                return attachmentId;
            }

            public void setAttachmentId(String attachmentId) {
                this.attachmentId = attachmentId;
            }

            public String getFileName() {
                return fileName;
            }

            public void setFileName(String fileName) {
                this.fileName = fileName;
            }

            public String getFileData() {
                return fileData;
            }

            public void setFileData(String fileData) {
                this.fileData = fileData;
            }

            public Date getDate() {
                return date;
            }

            public void setDate(Date date) {
                this.date = date;
            }

            @Override
            public boolean equals(Object o) {
                if (this == o) return true;
                if (o == null || getClass() != o.getClass()) return false;

                Attachment that = (Attachment) o;

                if (id != null ? !id.equals(that.id) : that.id != null) return false;
                if (userId != null ? !userId.equals(that.userId) : that.userId != null) return false;
                if (attachmentId != null ? !attachmentId.equals(that.attachmentId) : that.attachmentId != null) return false;
                if (fileName != null ? !fileName.equals(that.fileName) : that.fileName != null) return false;
                if (fileData != null ? !fileData.equals(that.fileData) : that.fileData != null) return false;
                if (date != null ? !date.equals(that.date) : that.date != null) return false;

            }

            @Override
            public int hashCode() {
                int result = id != null ? id.hashCode() : 0;
                result = 31 * result + (userId != null ? userId.hashCode() : 0);
                result = 31 * result + (attachmentId != null ? attachmentId.hashCode() : 0);
                result = 31 * result + (fileName != null ? fileName.hashCode() : 0);
                result = 31 * result + (fileData != null ? fileData.hashCode() : 0);
                result = 31 * result + (date != null ? date.hashCode() : 0);
                return result;
            }

            @Override
            public String toString() {
                return userFileName;
            }

        }

        public class AttachmentDataList implements Serializable {
            private ArrayList<Attachment> attList;

            public ArrayList<Attachment> getAttList() {
                return attList;
            }

            public void setAttList(ArrayList<Attachment> attList) {
                this.attList = attList;
            }
        }

- KJEjava48

---

请仔细检查您的代码，可能存在一个导致内存溢出错误的递归调用。 - ADM

或许你的 JSON 数据比 Gson 预期的要大得多 ;) - user2756345

是的，我同意Wizard和ADM的观点。我还没有查看完整的代码，但如果你正在缓存JSON数据，并且应用程序的内存不足，可能会发生此异常。或者你声明了过多的静态对象或数组对象。 - Developine

@LyubomyrShaydariv 我们将附加的图像保存为字符串，而不是作为文件上传到数据库中。因此，位图图像通过base64编码转换为字符串数据。 - KJEjava48

使用org.springframework.web.client.RestTemplate的resttemplate.exchange()方法。你是指这个吗？ - KJEjava48

Answer 1

您遇到了 OutOfMemoryError 的问题，因为您正在使用效率低下且占用大量内存的Base64转换。另一个问题是Gson：它没有为 JsonWriter 和 JsonReader 类提供任何原始写入方法：在这里，您最多只能写入 / 读取一个单独的字符串值。将大量输入收集到单个字符串中也是另一个非常消耗内存的操作：请检查您的堆栈跟踪以确保在非常底层使用了字符串构建器实例--这仅仅是为了将单个值写入输出流。简而言之，如果我没有误解您的代码（因为它似乎缺少真正重要的部分，所以我只是尝试重构您的场景）：

• 获取字节数组（这将是一个新对象，可能是另一个字节数组的克隆）；
• 将字节数组转换为Base64编码字符串（这也会影响性能，因为它会克隆字节数组以创建防御性副本）；
• 将所有内容转换为字符串gson.toJson(attDataAcc, AttachmentDataList.class);-- 又是一个巨大的开销。

所有这些都非常消耗内存。如果Gson能够支持向输出流进行原始写入，那就太好了，但目前它还没有任何支持。理论上，您可以通过直接向底层流（可能是从您的字节数组源直接进行而不进行任何大规模转换，因为Base64也可以作为流进行传输，从而最小化内存消耗）进行写入来克服此问题。您提到了Gson 2.6.2，但我正在使用Gson 2.8.0，因此下面的解决方案只能在Gson 2.8.0中100％工作，并且甚至可能不适用于任何其他较小的Gson版本，因为它使用反射来“hack” JsonWriter类。

final class ByteArrayTypeAdapter
        extends TypeAdapter<byte[]> {

    // These two methods and one field from the super class privates are necessary to make it all work  
    private static final Method writeDeferredNameMethod;
    private static final Method beforeValueMethod;
    private static final Field writerField;

    static {
        try {
            writeDeferredNameMethod = JsonWriter.class.getDeclaredMethod("writeDeferredName");
            writeDeferredNameMethod.setAccessible(true);
            beforeValueMethod = JsonWriter.class.getDeclaredMethod("beforeValue");
            beforeValueMethod.setAccessible(true);
            writerField = JsonWriter.class.getDeclaredField("out");
            writerField.setAccessible(true);
        } catch ( final NoSuchMethodException | NoSuchFieldException ex ) {
            throw new RuntimeException(ex);
        }
    }

    // This type adapter is effectively a singleton having no any internal state
    private static final TypeAdapter<byte[]> byteArrayTypeAdapter = new ByteArrayTypeAdapter();

    private ByteArrayTypeAdapter() {
    }

    // But making the constructor private and providing access to the instance via the method, we make sure that the only instance exists and it's safe
    static TypeAdapter<byte[]> getByteArrayTypeAdapter() {
        return byteArrayTypeAdapter;
    }

    @Override
    public void write(final JsonWriter out, final byte[] bytes)
            throws IOException {
        try {
            // Since we're writing a byte[] array, that's probably a field value, make sure that the corresponding property name has been written to the output stream
            writeDeferredNameAndFlush(out);
            // Now simulate JsonWriter.value(byte[]) if such a method could exist
            writeRawBase64ValueAndFlush(bytes, (Writer) writerField.get(out));
        } catch ( IllegalAccessException | InvocationTargetException ex ) {
            throw new IOException(ex);
        }
    }

    @Override
    public byte[] read(final JsonReader in) {
        // If necessary, requires more hacks...
        // And this is crucial for the server-side:
        // In theory, the client can generate HUGE Base64 strings,
        // So the server could crash with OutOfMemoryError too
        throw new UnsupportedOperationException();
    }

    private static void writeDeferredNameAndFlush(final Flushable out)
            throws IOException, IllegalAccessException, InvocationTargetException {
        writeDeferredNameMethod.invoke(out);
        beforeValueMethod.invoke(out);
        // Flush is necessary: the JsonWriter does not know that we're using its private field intruding to its privates and may not flush
        out.flush();
    }

    private static void writeRawBase64ValueAndFlush(final byte[] bytes, final Writer writer)
            throws IOException {
        // Writing leading "
        writer.write('\"');
        // This comes from Google Guava
        final BaseEncoding baseEncoding = BaseEncoding.base64();
        final OutputStream outputStream = baseEncoding.encodingStream(writer);
        // This too
        // Note that we just r_e_d_i_r_e_c_t streams on fly not making heavy transformations
        ByteStreams.copy(new ByteArrayInputStream(bytes), outputStream);
        // This is necessary too
        outputStream.close();
        // Writing trailing "
        writer.write('\"');
        // Flush again to keep it all in sync
        writer.flush();
    }

}

我知道这是一种hack的方式，但它比不断遇到OutOfMemoryError要好。

现在，只需让它与Spring RestTemplates一起工作：

// Gson is thread-safe and can be re-used
private static final Gson gson = new GsonBuilder()
        // SimpleDateFormat may be NOT thread-safe so you should not share the single SimpleDateFormat between threads
        // However Gson supports date/time formats out of box
        .setDateFormat("yyyy-MM-dd HH:mm:ss")
        // Registering byte[] to the type adapter
        .registerTypeAdapter(byte[].class, getByteArrayTypeAdapter())
        .create();

private static final RestTemplate restTemplate = new RestTemplate();
private static final String URL = "http://localhost";

public static void main(final String... args) {
    sendPostRequest("hello world".getBytes(), byte[].class);
}

private static void sendPostRequest(final Object object, final Type type) {
    // This is where we're binding the output stream I was asking in the question comments
    final RequestCallback requestCallback = request -> gson.toJson(object, type, new OutputStreamWriter(request.getBody()));
    // Spring RestTemplates stuff here...
    final SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
    requestFactory.setBufferRequestBody(false);
    final ResponseExtractor<String> responseExtractor = new HttpMessageConverterExtractor<>(String.class, restTemplate.getMessageConverters());
    restTemplate.setRequestFactory(requestFactory);
    // Let it fly
    restTemplate.execute(URL, POST, requestCallback, responseExtractor);
}

注意，您可能需要编写专门的类型适配器来处理某些特殊类型，使其能够直接写入输出流，以便完全消除byte[]。您还可以在官方Gson问题跟踪器上为此问题投票：https://github.com/google/gson/issues/971 ，也许在未来版本的Gson中就不再需要使用Java反射API了。

- Lyubomyr Shaydariv