Why does node data close when hadoop starts?

Question

I have hasoop 1.0.4 installed on ubuntu 11.0.4 in VirtualBox (same as my hostname), since the node data terminates giving the following error in the log files

/************************************************************ STARTUP_MSG: Starting DataNode STARTUP_MSG: host = VirtualBox/127.0.1.1 STARTUP_MSG: args = [] STARTUP_MSG: version = 1.0.4 STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290; compiled by 'hortonfo' on Wed Oct 3 05:13:58 UTC 2012 ************************************************************/ 2013-08-18 19:52:21,301 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2013-08-18 19:52:21,394 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered. 2013-08-18 19:52:21,412 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2013-08-18 19:52:21,417 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started 2013-08-18 19:52:23,207 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered. 2013-08-18 19:52:23,276 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists! 2013-08-18 19:52:26,887 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Incompatible namespaceIDs in /app/hadoop/tmp/dfs/data: namenode namespaceID = 457871323; datanode namespaceID = 2066655210 at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:232) at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:147) at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:385) at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:299) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1582) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1521) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1539) at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1665) at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1682) 2013-08-18 19:52:26,903 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down DataNode at VirtualBox/127.0.1.1 ************************************************************/ 

Any idea why? How can I fix it?

Answer 1

It happened to me several times. If restarting the node data does not help, follow these steps:

• Restart Hadoop
• Go to / app / hadoop / tmp / dfs / name / current
• Open VERSION (i.e. vim VERSION )
• NamespaceID entry
• Go to / app / hadoop / tmp / dfs / data / current
• Open VERSION (i.e. vim VERSION )
• Replace the namespace identifier with the namespace identifier you recorded in step 4.

This should fix the problem.

Answer 2

This is a well-known issue when formatting a node name. Please refer below:

http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/#javaioioexception-incompatible-namespaceids

Answer 3

This usually occurs when you format the namenode and do not clear the datanode and namespace changes.

Possible Solution:

Try deleting the directory / app / hadoop / tmp / dfs / data strong> and restart datanode

Answer 4

I lost my data in namenode. I had to format my namenode with

 hadoop namenode -format 

After running namenode, datanode did not start due to a problem with VERSION

Then I need to also delete all data from datanodes.

I can just empty the data folder in dfs (/{hadoop.tmp.dirasket/dfs/data/*) and restart the datanode

ie rm -rf /{hadoop.tmp.dir}/dfs/data/*

OR

delete the VERSION file (/{hadoop.tmp.dirasket/dfs/data/current/VERSION) and restart the datanode

Answer 5

 Restart Hadoop Go to /app/hadoop/tmp/dfs/name/current Open VERSION (ie by vim VERSION) Record namespaceID Go to /app/hadoop/tmp/dfs/data/current Open VERSION (ie by vim VERSION) Replace the namespaceID with thve namespaceID you recorded in step 4. 

To do this above, I do not have files in the data folder / tmp / data p>