Hadoop FileSystem instance animations

Question

I have a class (I remove try / catch for readability):

public class HadoopFileSystem { private FileSystem m_fileSystem = null; public HadoopFileSystem() { Configuration l_configuration = new Configuration(); l_configuration .set("fs.default.name", "hdfs://localhost:9100"); l_configuration .set("mapred.job.tracker", "localhost:9101"); m_fileSystem = FileSystem.get(l_configuration ); } public void close() { m_fileSystem.close(); } public void createFile(String a_pathDFS) { m_fileSystem.create(new Path(a_pathDFS)); } } 

In my program, I am the first HadoopFileSysem object, I do not close it .

Then I create a second HadoopFileSysem object and I close it.

Finally, when I want to use the function in m_fileSystem in my first object, I have an error: java.io.IOException: Filesystem closed

But I did not close it!

Here is some code to illustrate my problem:

 HadoopFileSystem h1 = new HadoopFileSystem(); HadoopFileSystem h2 = new HadoopFileSystem(); if(h1 == h2) System.out.println("=="); // No print if(h1.equals(h2)) System.out.println("equals"); // No print h2.close(); h1.createFile("test.test"); // ERROR : java.io.IOException: Filesystem closed h1.close(); 

Why?

Answer 1

m_fileSystem = FileSystem.get(l_configuration ); is a static call, even if you have two different objects. You need to find a way to not make this call static for two different objects.

Try this to end the problem,

 conf.setBoolean("fs.hdfs.impl.disable.cache", true); 

Answer 2

You create an object based on a hard-coded configuration. This basically means that you create 2 identical objects. Since these objects are identical, the JVM will refer to the same object. Thus, h1 and h2 refer to the same object.

The reason for this is because you are getting an existing instance of the object based on the configuration file. If the configuration is different for h1 and h2, this will not be the same instance.