如何对大型XML文件进行XSD模式验证？

Question

我需要在有限的内存使用情况下验证大型 XML。到目前为止，我找到的每种方法都会出现内存不足错误。

我尝试过的方法：

 //method 1
        SAXParserFactory factory = SAXParserFactory.newInstance();
        factory.setValidating(false);
        factory.setNamespaceAware(true);

        SchemaFactory schemaFactory = SchemaFactory.newInstance("http://www.w3.org/2001/XMLSchema");
        factory.setSchema(schemaFactory.newSchema(new Source[] {new StreamSource(Thread.currentThread().getContextClassLoader().getResource("xmlresource/XSD_final2.xsd").getFile())}));
        SAXParser parser = factory.newSAXParser();
        XMLReader reader = parser.getXMLReader();
        reader.setErrorHandler(new SimpleErrorHandler());
        reader.parse(new InputSource(inputXml));
//method2 

XMLValidationSchemaFactory sf = XMLValidationSchemaFactory.newInstance(XMLValidationSchema.SCHEMA_ID_W3C_SCHEMA);
            XMLValidationSchema vs = sf.createSchema(Thread.currentThread().getContextClassLoader().getResource("xmlresource/XSD_final2.xsd"));
            XMLStreamReader2 sr = (XMLStreamReader2) XMLInputFactory2.newInstance().createXMLStreamReader(new FileInputStream(inputXml));
            sr.validateAgainst(vs);
            try {
              while (sr.hasNext()) {
                sr.next();
              }
              System.out.println("Validated ok!");
            } catch (XMLValidationException ve) {
              System.err.println("Validation problem: "+ve);
              isValid = false;
            }
            sr.close();

//方法3

      SchemaFactory factory = SchemaFactory.newInstance("http://www.w3.org/2001/XMLSchema");
          String fileName = Thread.currentThread().getContextClassLoader().getResource("xmlresource/XSD_final2.xsd").getFile();

          Schema schema = factory.newSchema(new File(fileName));
          Validator validator = schema.newValidator();

          // create a source from a file
          StreamSource source = new StreamSource(new File(inputXml));

          // check input

            validator.validate(source);

每次都出现内存溢出错误。

编辑：

使用XOM。

SAXParserFactory factory = SAXParserFactory.newInstance();
            factory.setValidating(false);
            factory.setNamespaceAware(true);

            SchemaFactory schemaFactory = SchemaFactory.newInstance("http://www.w3.org/2001/XMLSchema");
            factory.setSchema(schemaFactory.newSchema(new Source[] {new StreamSource(Thread.currentThread().getContextClassLoader().getResource("xmlresource/XSD_final2.xsd").getFile())}));
            SAXParser parser = factory.newSAXParser();
            XMLReader reader = parser.getXMLReader();
            reader.setErrorHandler(new SimpleErrorHandler());

            Builder builder = new Builder(reader);
            builder.build(new FileInputStream(new File(inputXml)));

尽管XML文件只有15MB，但内存使用率仍然非常高，达到了250MB。

堆栈跟踪：

Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2367)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:535)
at java.lang.StringBuffer.append(StringBuffer.java:322)
at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.handleCharacters(XMLSchemaValidator.java:1574)
at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.characters(XMLSchemaValidator.java:789)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:441)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:835)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:764)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:123)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1210)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:568)
at nu.xom.Builder.build(Unknown Source)
at nu.xom.Builder.build(Unknown Source)

编辑 我的XML中有一个很大的Base64字符串。

- bunnyjesse112

Answer 1

请看Marco Tedone关于XML解组的文章（点击此处）。根据他的结论，我建议使用STax以实现低内存消耗。

    XMLInputFactory xmlInputFactory = XMLInputFactory.newInstance();
    XMLStreamReader xmlStreamReader = xmlInputFactory.createXMLStreamReader(fileInputStream);
    Validator validator = schema.newValidator();
    validator.validate(new StAXSource(xmlStreamReader));

- remi dupuis

---

感谢您的回复。这仍然使用xerces，所以我仍然会在-Xmx250m下遇到内存溢出问题。到目前为止，woodstox对我来说效果最好。 - bunnyjesse112

Answer 2

可能是内存被模式所使用，而不是源文档。您没有提及模式的任何信息。有些模式可能会使用非常高的内存，例如如果您在内容模型中具有大的minOccurs或maxOccurs有限值。内存不足异常是在什么时候发生的？

- Michael Kay

---

感谢您的回复。Xsd 有一些最小/最大出现次数，但并不复杂。我的 XML 包含 Base64 字符串，并且在 AbstractStringBuilder 中看到了内存溢出。 - bunnyjesse112