logo标签列表

启动Hadoop Namenode时出现错误。

hadoopconfigurationportstartup
4
我希望在我的Ubuntu机器上实现一个伪分布式的Hadoop系统。但是我无法启动namenode（其他如jobtracker可以正常启动）。 我的启动命令是：
./hadoop namenode -format
./start-all.sh

我检查了位于logs/hadoop-mongodb-namenode-mongodb.log的namenode日志。
 65 2013-12-25 13:44:39,797 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: ReplicateQueue QueueProcessingStatistics: Queue flush completed 0 blocks in 0 msec processing time, 0 msec cl    ock time, 1 cycles
 66 2013-12-25 13:44:39,797 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: InvalidateQueue QueueProcessingStatistics: First cycle completed 0 blocks in 0 msec
 67 2013-12-25 13:44:39,797 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: InvalidateQueue QueueProcessingStatistics: Queue flush completed 0 blocks in 0 msec processing time, 0 msec c    lock time, 1 cycles
 68 2013-12-25 13:44:39,799 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source FSNamesystemMetrics registered.
 69 2013-12-25 13:44:39,809 INFO org.apache.hadoop.ipc.Server: Starting SocketReader
 70 2013-12-25 13:44:39,810 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcDetailedActivityForPort9000 registered.
 71 2013-12-25 13:44:39,810 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcActivityForPort9000 registered.
 72 2013-12-25 13:44:39,812 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at: localhost/127.0.0.1:9000
 73 2013-12-25 13:44:39,847 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
 74 2013-12-25 13:44:39,878 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
 75 2013-12-25 13:44:39,884 INFO org.apache.hadoop.http.HttpServer: dfs.webhdfs.enabled = false
 76 2013-12-25 13:44:39,888 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 50070
 77 2013-12-25 13:44:39,889 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:mongodb cause:java.net.BindException: Address already in use
 78 2013-12-25 13:44:39,889 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: ReplicationMonitor thread received InterruptedExceptionjava.lang.InterruptedException: sleep interrupted
 79 2013-12-25 13:44:39,890 INFO org.apache.hadoop.hdfs.server.namenode.DecommissionManager: Interrupted Monitor
 80 java.lang.InterruptedException: sleep interrupted
 81     at java.lang.Thread.sleep(Native Method)
 82     at org.apache.hadoop.hdfs.server.namenode.DecommissionManager$Monitor.run(DecommissionManager.java:65)
 83     at java.lang.Thread.run(Thread.java:701)
 84 2013-12-25 13:44:39,890 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 0 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number o    f syncs: 0 SyncTimes(ms): 0
 85 2013-12-25 13:44:39,905 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: closing edit log: position=4, editlog=/var/hadoop/hadoop-1.2.1/dfs.name.dir/current/edits
 86 2013-12-25 13:44:39,905 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: close success: truncate to 4, editlog=/var/hadoop/hadoop-1.2.1/dfs.name.dir/current/edits
 87 2013-12-25 13:44:39,909 INFO org.apache.hadoop.ipc.Server: Stopping server on 9000
 88 2013-12-25 13:44:39,909 INFO org.apache.hadoop.ipc.metrics.RpcInstrumentation: shut down
 89 2013-12-25 13:44:39,909 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: java.net.BindException: Address already in use
 90     at sun.nio.ch.Net.bind0(Native Method)
 91     at sun.nio.ch.Net.bind(Net.java:174)
 92     at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:139)
 93     at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:77)
 94     at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
 95     at org.apache.hadoop.http.HttpServer.start(HttpServer.java:602)
 96     at org.apache.hadoop.hdfs.server.namenode.NameNode$1.run(NameNode.java:517)
 97     at org.apache.hadoop.hdfs.server.namenode.NameNode$1.run(NameNode.java:395)
 98     at java.security.AccessController.doPrivileged(Native Method)
 99     at javax.security.auth.Subject.doAs(Subject.java:416)
100     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
101     at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:395)
102     at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:337)
103     at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:569)
104     at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1479)
105     at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1488)
106 
107 2013-12-25 13:44:39,910 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
108 /************************************************************
109 SHUTDOWN_MSG: Shutting down NameNode at mongodb/192.168.10.2
110 ************************************************************/
                                                                                                                                                                                 110,1         Bot
 63 2013-12-25 13:44:39,796 INFO org.apache.hadoop.util.HostsFileReader: Refreshing hosts (include/exclude) list
 64 2013-12-25 13:44:39,796 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: ReplicateQueue QueueProcessingStatistics: First cycle completed 0 blocks in 0 msec
 65 2013-12-25 13:44:39,797 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: ReplicateQueue QueueProcessingStatistics: Queue flush completed 0 blocks in 0 msec processing time, 0 msec cl    ock time, 1 cycles
 66 2013-12-25 13:44:39,797 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: InvalidateQueue QueueProcessingStatistics: First cycle completed 0 blocks in 0 msec
 67 2013-12-25 13:44:39,797 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: InvalidateQueue QueueProcessingStatistics: Queue flush completed 0 blocks in 0 msec processing time, 0 msec c    lock time, 1 cycles
 68 2013-12-25 13:44:39,799 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source FSNamesystemMetrics registered.
 69 2013-12-25 13:44:39,809 INFO org.apache.hadoop.ipc.Server: Starting SocketReader
 70 2013-12-25 13:44:39,810 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcDetailedActivityForPort9000 registered.
 71 2013-12-25 13:44:39,810 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcActivityForPort9000 registered.
 72 2013-12-25 13:44:39,812 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at: localhost/127.0.0.1:9000
 73 2013-12-25 13:44:39,847 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
 74 2013-12-25 13:44:39,878 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
 75 2013-12-25 13:44:39,884 INFO org.apache.hadoop.http.HttpServer: dfs.webhdfs.enabled = false
 76 2013-12-25 13:44:39,888 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 50070
 77 2013-12-25 13:44:39,889 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:mongodb cause:java.net.BindException: Address already in use
 78 2013-12-25 13:44:39,889 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: ReplicationMonitor thread received InterruptedExceptionjava.lang.InterruptedException: sleep interrupted
 79 2013-12-25 13:44:39,890 INFO org.apache.hadoop.hdfs.server.namenode.DecommissionManager: Interrupted Monitor
 80 java.lang.InterruptedException: sleep interrupted
 81     at java.lang.Thread.sleep(Native Method)
 82     at org.apache.hadoop.hdfs.server.namenode.DecommissionManager$Monitor.run(DecommissionManager.java:65)
 83     at java.lang.Thread.run(Thread.java:701)
 84 2013-12-25 13:44:39,890 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 0 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number o    f syncs: 0 SyncTimes(ms): 0
 85 2013-12-25 13:44:39,905 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: closing edit log: position=4, editlog=/var/hadoop/hadoop-1.2.1/dfs.name.dir/current/edits
 86 2013-12-25 13:44:39,905 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: close success: truncate to 4, editlog=/var/hadoop/hadoop-1.2.1/dfs.name.dir/current/edits
 87 2013-12-25 13:44:39,909 INFO org.apache.hadoop.ipc.Server: Stopping server on 9000
 88 2013-12-25 13:44:39,909 INFO org.apache.hadoop.ipc.metrics.RpcInstrumentation: shut down
 89 2013-12-25 13:44:39,909 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: java.net.BindException: Address already in use
 90     at sun.nio.ch.Net.bind0(Native Method)
 91     at sun.nio.ch.Net.bind(Net.java:174)
 92     at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:139)
 93     at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:77)
 94     at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
 95     at org.apache.hadoop.http.HttpServer.start(HttpServer.java:602)
 96     at org.apache.hadoop.hdfs.server.namenode.NameNode$1.run(NameNode.java:517)
 97     at org.apache.hadoop.hdfs.server.namenode.NameNode$1.run(NameNode.java:395)
 98     at java.security.AccessController.doPrivileged(Native Method)
 99     at javax.security.auth.Subject.doAs(Subject.java:416)
100     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
101     at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:395)
102     at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:337)
103     at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:569)
104     at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1479)
105     at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1488)
106 
107 2013-12-25 13:44:39,910 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:

这是错误信息。很明显，端口号出错了！以下是我的配置文件：

core-site.xml
  1 <?xml version="1.0"?>
  2 <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
  3 <configuration>
  4 <property>
  5 <name>fs.default.name</name>
  6 <value>hdfs://localhost:9000</value>
  7 </property>
  8 </configuration>

hdfs-site.xml

  1 <?xml version="1.0"?>
  2 <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
  3 
  4 <!-- Put site-specific property overrides in this file. -->
  5 <configuration>
  6      <property>
  7               <name>dfs.replication</name>
  8                <value>1</value>
  9      </property>
 10 
 11      <property>
 12              <name>dfs.name.dir</name>
 13              <value>/var/hadoop/hadoop-1.2.1/dfs.name.dir</value>
 14      </property>
 15 
 16      <property>
 17              <name>dfs.data.dir</name>
 18              <value>/var/hadoop/hadoop-1.2.1/dfs.data.dir</value>
 19      </property>
 20 </configuration>

无论我将端口更改为其他端口并重新启动Hadoop，错误仍然存在！有人可以帮我吗？
执行此命令后，jps 命令的输出是什么？ - Mehraban
你好，我使用jps命令，它显示了SecondaryNamenode和Datanode，所以我杀掉了所有它显示的进程。但错误仍然存在... - wuchang
2个回答
2

尝试删除hdfs数据目录，而不是在启动hdfs之前格式化namenode，先启动hdfs并检查jps输出。如果一切正常，则尝试格式化namenode并重新检查。如果仍然有问题，请给我日志详细信息。

P.S: 不要杀死进程。只需使用stop-all.sh或任何你应该使用的方法停止Hadoop。

我按照你告诉我的做了。我使用命令start-dfs.sh启动了hdfs，然后使用jps检查，结果显示DataNode进程和SecondaryNameNode进程已经启动了，但没有其他进程。接下来，我格式化了hdfs，但jps的输出并没有改变！ - wuchang
NameNode没有启动，因此应该有一个包含错误的日志文件！ - Mehraban
我按照你说的做了，就一样。在启动hadoop之前，我删除了数据目录。 - wuchang
请删除所有日志文件，再重复一遍，并再次查看namenode日志？我只想确定异常情况。 - Mehraban
嗨，我尝试了所有的方法，包括删除整个Hadoop目录并从官方网站下载的JAR文件中提取一个新的目录，但问题仍然存在。 - wuchang
显示剩余6条评论
0

我的集群中一个从节点上的数据节点出现了与端口绑定类似的异常：

ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.net.BindException: Address already in use

我注意到数据节点的默认Web界面端口即50075已经绑定到另一个应用程序：

[ap2]-> netstat -an | grep -i 50075
tcp        0      0 10.0.1.1:45674              10.0.1.1:50075              ESTABLISHED
tcp        0      0 10.0.1.1:50075              10.0.1.1:45674              ESTABLISHED
[ap2]->

我已经在conf/hdfs-site.xml中更改了Datanode的Web界面：

<property>
<name>dfs.datanode.http.address</name>
<value>10.0.1.1:50080</value>
<description>Datanode http port</description>
</property>

这有助于解决问题，同样您可以通过在conf/hadoop-site.xml中设置dfs.http.address来更改Web界面侦听的默认地址和端口，例如localhost:9090，但请确保该端口可用。

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文，
原文链接