Janusgraph OLAP遍历-使用信任存储配置与Cassandra建立连接不起作用

3
我们已经成功地配置了Janusgraph 0.5.2版本,我们能够根据需求插入和查询(OLTP)数据。 我们正在研究JanusGraph OLAP遍历以满足一些报告和分析需求。 然而,当我尝试按照JanusGraph文档提供的说明操作时,我们无法连接到Cassandra,当我们尝试遍历图形时发生问题。 Cassandra在客户端上设置有SSL连接Truststore。这种配置从Gremlin Console连接到OLTP遍历的Cassandra很好用。
以下是OLTP的配置,它可以使用janusgraph-cql-oltp.properties
gremlin.graph=org.janusgraph.core.JanusGraphFactory
storage.backend=cql
storage.hostname=cassandra.cassandra.svc.cluster.local
storage.username=cassandra
storage.password=cassandra123
storage.cql.keyspace=janusgraph
cache.db-cache = true
cache.db-cache-clean-wait = 20
cache.db-cache-time = 180000
cache.db-cache-size = 0.5
storage.lock.wait-time = 60000
storage.cql.ssl.enabled=true
storage.cql.ssl.truststore.location=/etc/config/tls/gremlin/client/truststore
storage.cql.ssl.truststore.password=secretpasswd

当我在Gremlin控制台中加载此行以连接和遍历简单查询时,我能够成功执行。
以下是OLAP的配置,显示连接到Cassandra时出现错误:
gremlin.graph=org.apache.tinkerpop.gremlin.hadoop.structure.HadoopGraph
gremlin.hadoop.graphReader=org.janusgraph.hadoop.formats.cql.CqlInputFormat
gremlin.hadoop.graphWriter=org.apache.hadoop.mapreduce.lib.output.NullOutputFormat

gremlin.hadoop.jarsInDistributedCache=true
gremlin.hadoop.inputLocation=none
gremlin.hadoop.outputLocation=output
gremlin.spark.persistContext=true
# # JanusGraph Cassandra InputFormat configuration
# # These properties defines the connection properties which were used while write data to JanusGraph.
janusgraphmr.ioformat.conf.storage.backend=cql
# This specifies the hostname & port for Cassandra data store.
janusgraphmr.ioformat.conf.storage.hostname=cassandra.cassandra.svc.cluster.local
janusgraphmr.ioformat.conf.storage.port=9042
janusgraphmr.ioformat.conf.storage.username=cassandra
janusgraphmr.ioformat.conf.storage.password=cassandra123
janusgraphmr.ioformat.conf.storage.cql.keyspace=janusgraph
janusgraphmr.ioformat.conf.storage.lock.wait-time = 60000
janusgraphmr.ioformat.conf.storage.cql.ssl.enabled=true
janusgraphmr.ioformat.conf.storage.cql.ssl.truststore.location=/etc/config/tls/gremlin/client/truststore
janusgraphmr.ioformat.conf.storage.cql.ssl.truststore.password=cassandra123

janusgraphmr.ioformat.conf.storage.ssl.enabled=true
janusgraphmr.ioformat.conf.storage.ssl.truststore.location=/etc/config/tls/gremlin/client/truststore
janusgraphmr.ioformat.conf.storage.ssl.truststore.password=cassandra123

janusgraphmr.ioformat.conf.storage.cql.read-consistency-level=ONE

storage.lock.wait-time = 60000
storage.cql.ssl.enabled=true
storage.cql.ssl.client-authentication-enabled=true
storage.cql.ssl.truststore.location=/etc/config/tls/gremlin/client/truststore
storage.cql.ssl.truststore.password=cassandra123

janusgraphmr.ioformat.conf.cache.db-cache = true
janusgraphmr.ioformat.conf.cache.db-cache-clean-wait = 20
janusgraphmr.ioformat.conf.cache.db-cache-time = 180000
janusgraphmr.ioformat.conf.cache.db-cache-size = 0.5

cassandra.input.partitioner.class=org.apache.cassandra.dht.Murmur3Partitioner
cassandra.input.widerows=true

# # SparkGraphComputer Configuration #
spark.master=local[*]
spark.executor.memory=1g
spark.serializer=org.apache.spark.serializer.KryoSerializer
spark.kryo.registrator=org.janusgraph.hadoop.serialize.JanusGraphKryoRegistrator

当我在Gremlin控制台中加载图形对象时,我可以看到属性已正确加载。 但是当我按照文档中所述遍历图形时,我会遇到与SSL配置相关的Cassandra连接错误。

gremlin> graph=HadoopGraph.open('/janusgraph-full-0.5.2/conf/olap.properties')
==>hadoopgraph[cqlinputformat->nulloutputformat]
gremlin> g=graph.traversal().withComputer(SparkGraphComputer)
==>graphtraversalsource[hadoopgraph[cqlinputformat->nulloutputformat], sparkgraphcomputer]
gremlin> graph.configuration()
//// i can see all the properties from the file loaded here
gremlin> g.V().limit(1)
07:34:44 WARN  org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer  - class org.apache.hadoop.mapreduce.lib.output.NullOutputFormat does not implement PersistResultGraphAware and thus, persistence options are unknown -- assuming all options are possible
com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: cassandra.cassandra.svc.cluster.local/10.0.165.158:9042 (com.datastax.driver.core.exceptions.TransportException: [cassandra.cassandra.svc.cluster.local/10.0.165.158:9042] Connection has been closed))
Type ':help' or ':h' for help.

我可以从我的Cassandra日志中验证,一个连接尝试了但是由于SSL原因被拒绝了请求。以下是来自Cassandra实例的日志:

INFO  [epollEventLoopGroup-2-4] 2023-05-02 07:34:58,809 Message.java:826 - Unexpected exception during request; channel = [id: 0xeb0e017f, L:/10.12.0.224:9042 ! R:/10.12.0.135:60316]
io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 0400000001000000500003000b43514c5f56455253494f4e0005332e302e30000e4452495645525f56455253494f4e0005332e392e30000b4452495645525f4e414d4500144461746153746178204a61766120447269766572
        at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1057) ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411) [netty-all-4.0.44.Final.jar:4.0.44.Final]

有人能帮忙解决如何将SSL配置传递给Gremlin,以便在JanusGraph上进行OLAP遍历吗?

1个回答

4
你可以尝试使用以下配置选项吗?
cassandra.input.native.ssl.trust.store.path
cassandra.input.native.ssl.trust.store.password

You can find how they are used from CqlConfigHelper.java. Unfortunately, the official doc does not mention this.


1
我尝试了上述选项,但现在出现了不同的错误,表明它接受了提供的详细信息,但在连接过程中进一步失败了。错误详细信息指向运行时和编译时使用的jar包版本不匹配。错误是 - Caused by: java.lang.NoSuchFieldError: DEFAULT_SSL_CIPHER_SUITES         at org.apache.cassandra.hadoop.cql3.CqlConfigHelper.getSSLOptions(CqlConfigHelper.java:533)         at org.apache.cassandra.hadoop.cql3.CqlConfigHelper.getCluster(CqlConfigHelper.java:315)        ..... .... - blankCoder
3
我刚刚注意到你正在使用0.5.2版本。你能否尝试一下0.6或1.0版本?0.5版本已经不再得到官方支持,而且在0.6版本中有很多变化。理论上,你可以继续在0.5上使用OLTP,同时在0.6/1.0上运行OLAP。 - Boxuan Li
1
如果使用0.6版本无法正常工作,您可能需要跟进此问题:https://github.com/JanusGraph/janusgraph/issues/1727 - Boxuan Li
3
谢谢您的输入。我已使用0.5.2在本地重新创建了问题,并验证了Janusgraph的最新版本已经解决了这个问题。 - blankCoder

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接