All geek questions in one place

Is there a way to upload an HDFS file using the WebHDFS REST API?

Is there any way I can download a file from HDFS using the WebHDFS REST API? The closest I came to is to use an open operation to read a file and save the contents.

curl -i -L "http://localhost:50075/webhdfs/v1/demofile.txt?op=OPEN" -o ~/demofile.txt

Is there any API that allows me to download a file directly without opening it? I looked through the white paper and tried Google as well, but couldn't find anything. Can someone point me in the right direction or provide me with a few pointers?

Thank you so much for your precious time.

+6
source share
1 answer

Perhaps you can use the DataNode API for this (by default on port 50075), it supports the streamFile command, which you could use. Using wget , it looks something like this:

 wget http://$datanode:50075/streamFile/demofile.txt -O ~/demofile.txt

Please note that this command should be executed on the datanode itself, and not on nenenode!

Alternatively, if you don’t know which datanode gets you, you can set jobtracker and it will redirect you to the right datanode using this URL:

 http://$namenode:50070/data/demofile.txt
+6
source

Source: https://habr.com/ru/post/946291/

More articles:


All Articles