All geek questions in one place

About renaming the hoods huds file system

I store a lot of data in hdf. And I need to move files from one folder to another.

Can anyone ask how much the method of renaming a file system costs? Let's say I need to move terabytes of data.

Thank you very much.

+2
source share
2 answers

Moving files to HDFS or any file system, if implemented, involves changing the namespace and not moving the actual data. Looking through the code, only changes in the namespace (memory and editing log) in the node name are made.

From NameNode.java class

  • NameNode manages two critical tables:
  • 1) filename-> blocksequence (namespace)
  • 2) block-> machine translation ("inodes")

Only the first part needs to be changed; there is no need to block the list of machines. I have not tried, but I think everything should be OK.

+3
source

Renaming is an operation only for metadata in HDFS. Therefore, it is very cheap, as in the regular POSIX file system. Data is not moving. The only server is the name.

The source code for renaming can be found here . Pretty straight forward.

+3
source

Source: https://habr.com/ru/post/971926/

More articles:


All Articles