Giraph作业永远不会结束

3

我正在尝试使用最新的Giraph代码和Hadoop 2.5.2运行SimpleShortestPathsComputation示例。我的命令行如下:

hadoop jar /home/prhodes/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.5.2-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/prhodes/input/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/prhodes/giraph_output/shortestpaths -w 4

工作似乎开始得不错。但接着它开始输出这些类型的信息,而且这只会继续下去(看起来)直到你使用ctrl+c停止。

15/03/11 02:54:31 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
305.43 secs 15/03/11 02:54:31 INFO yarn.GiraphYarnClient: appattempt_1426041786848_0002_000001, State: ACCEPTED, Containers
used: 1 15/03/11 02:54:35 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
309.44 secs 15/03/11 02:54:35 INFO yarn.GiraphYarnClient: appattempt_1426041786848_0002_000001, State: ACCEPTED, Containers
used: 1 15/03/11 02:54:39 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
313.45 secs 15/03/11 02:54:39 INFO yarn.GiraphYarnClient: appattempt_1426041786848_0002_000001, State: ACCEPTED, Containers
used: 1 15/03/11 02:54:43 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
317.45 secs 15/03/11 02:54:43 INFO yarn.GiraphYarnClient: appattempt_1426041786848_0002_000001, State: ACCEPTED, Containers
used: 1 ^C15/03/11 02:54:47 INFO yarn.GiraphYarnClient: Giraph:
org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed:
321.46 secs 15/03/11 02:54:47 INFO yarn.GiraphYarnClient: appattempt_1426041786848_0002_000001, State: ACCEPTED, Containers
used: 1

你知道这里发生了什么吗?

1个回答

0
这事实际上是一个环境问题。我试图通过端口选择来将某些端口添加到我的 AWS 安全组白名单中,并错过了 YARN 在集群内部通信时使用的一些端口。我没有尝试整理整个列表,而是只添加了一个安全组规则,接受来自 10.0.0.0/24 的所有流量。这解决了这个问题。

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接