Running a job when moving a file in the hdfs directory (hadoop)

I want to run a task (hive or pig or oozie..etc) when the file is transferred to a specific directory (by tray) in the Hadoop distributed file system. Is it possible?

+4
source share
1 answer

This is possible indirectly. Oozie does not support data availability triggers. You need to set up a periodic flow with a certain frequency and add data availability as an additional condition.

This is a fairly common question about Oozi, unfortunately the documentation is poor.

+2
source

Source: https://habr.com/ru/post/1526528/


All Articles