Neo4j内存溢出问题

Question

Neo4j内存溢出问题

javawindowsneo4jout-of-memoryheap-memory

10

这是我从neo4j-apoc-1.0示例中获取的Main.java源代码。修改的目标是存储2个节点和1个关系的100万条记录：

package javaapplication2;

import org.neo4j.graphdb.GraphDatabaseService;
import org.neo4j.graphdb.Node;
import org.neo4j.graphdb.RelationshipType;
import org.neo4j.graphdb.Transaction;
import org.neo4j.kernel.EmbeddedGraphDatabase;


public class Main
{
    private static final String DB_PATH = "neo4j-store-1M";
    private static final String NAME_KEY = "name";

    private static enum ExampleRelationshipTypes implements RelationshipType
    {
        EXAMPLE
    }

    public static void main(String[] args)
    {
        GraphDatabaseService graphDb = null;

        try
        {
            System.out.println( "Init database..." );


            graphDb = new EmbeddedGraphDatabase( DB_PATH );

            registerShutdownHook( graphDb );


            System.out.println( "Start of creating database..." );


            int valIndex = 0;

            for(int i=0; i<1000; ++i)
            {
                for(int j=0; j<1000; ++j)
                {
                    Transaction tx = graphDb.beginTx();

                    try
                    {
                        Node firstNode = graphDb.createNode();
            firstNode.setProperty( NAME_KEY, "Hello" + valIndex );

                        Node secondNode = graphDb.createNode();
            secondNode.setProperty( NAME_KEY, "World" + valIndex );

                        firstNode.createRelationshipTo(
                           secondNode, ExampleRelationshipTypes.EXAMPLE );

                        tx.success();

                        ++valIndex;
                    }
                    finally
                    {
                        tx.finish();
                    }
                }
            }

            System.out.println("Ok, client processing finished!");
        }
        finally
        {
            System.out.println( "Shutting down database ..." );

            graphDb.shutdown();
        }
    }

    private static void registerShutdownHook( final GraphDatabaseService graphDb )
    {
        // Registers a shutdown hook for the Neo4j instance so that it
        // shuts down nicely when the VM exits (even if you "Ctrl-C" the
        // running example before it's completed)
        Runtime.getRuntime().addShutdownHook( new Thread()
        {
            @Override
            public void run()
            {
                graphDb.shutdown();
            }
        } );
    }
}

进行了几次迭代（大约150K次）后，我收到了以下错误信息：

"java.lang.OutOfMemoryError: Java heap space at java.nio.HeapByteBuffer.(HeapByteBuffer.java:39) at java.nio.ByteBuffer.allocate(ByteBuffer.java:312) at org.neo4j.kernel.impl.nioneo.store.PlainPersistenceWindow.(PlainPersistenceWindow.java:30) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.allocateNewWindow(PersistenceWindowPool.java:534) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.refreshBricks(PersistenceWindowPool.java:430) at org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool.acquire(PersistenceWindowPool.java:122) at org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.acquireWindow(CommonAbstractStore.java:459) at org.neo4j.kernel.impl.nioneo.store.AbstractDynamicStore.updateRecord(AbstractDynamicStore.java:240) at org.neo4j.kernel.impl.nioneo.store.PropertyStore.updateRecord(PropertyStore.java:209) at org.neo4j.kernel.impl.nioneo.xa.Command$PropertyCommand.execute(Command.java:513) at org.neo4j.kernel.impl.nioneo.xa.NeoTransaction.doCommit(NeoTransaction.java:443) at org.neo4j.kernel.impl.transaction.xaframework.XaTransaction.commit(XaTransaction.java:316) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.commit(XaResourceManager.java:399) at org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.commit(XaResourceHelpImpl.java:64) at org.neo4j.kernel.impl.transaction.TransactionImpl.doCommit(TransactionImpl.java:514) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:571) at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:543) at org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:102) at org.neo4j.kernel.EmbeddedGraphDbImpl$TransactionImpl.finish(EmbeddedGraphDbImpl.java:329) at javaapplication2.Main.main(Main.java:62) 28.05.2010 9:52:14 org.neo4j.kernel.impl.nioneo.store.PersistenceWindowPool logWarn WARNING: [neo4j-store-1M\neostore.propertystore.db.strings] Unable to allocate direct buffer"

各位大佬，求助！我做错了什么，该如何修复？在Windows XP 32位SP3平台上进行测试。也许可以通过创建自定义配置来解决问题？

感谢您提供的每个建议！

- Edward83

首先，将事务移动到外部循环可以大大提高速度。然后有批量插入器。您可以应用一些配置设置。但是，在这种情况下，我不知道是什么导致了错误（但似乎与Windows相关，在Linux上无法确认）。 - nawroth

1个回答

网页内容由stack overflow 提供, 点击上面的

可以查看英文原文，
原文链接

- Peter Neubauer · Accepted Answer

这是Windows上的配置问题，Neo4j无法使用内存映射缓冲区。相反，在堆上创建了一个Java缓冲区。在1.0版本中，默认情况下该缓冲区为470MB，超过了Windows JVM的默认堆大小。您有两个选择：

在pom.xml中切换到APOC 1.1-SNAPSHOT而不是1.0，它具有自动配置，将可用JVM堆的最大50％分配给Neo4j
通过运行Java来调整JVM堆更多（例如512Mb）

java -Xmx512m ....

您甚至可以在Eclipse的运行配置中将其插入JVM参数下

如果这样做有帮助，请告诉我们！

此外，对于每个节点对进行完整事务需要很长时间。尝试在第一个循环中打开事务，并仅在每1000个节点对之后提交？

/peter