All geek questions in one place

A simple example of a hive request?

I am trying to convert a simple workflow to oozie. I tried looking at oozie examples, but they are a bit overloaded. Effectively, I want to run the query and output the result to a text file.

hive -e 'select * from tables' > output.txt

How can I translate this into oozie so that it runs every hour?

+6
source share
1 answer

Your workflow might look something like this ... workflow.xml

 <workflow-app xmlns="uri:oozie:workflow:0.2" name="hive-wf"> <start to="hive-node"/> <action name="hive-node"> <hive xmlns="uri:oozie:hive-action:0.2"> <job-tracker>localhost:50001</job-tracker> <name-node>hdfs://localhost:50000</name-node> <configuration> <property> <name>mapred.job.queue.name</name> <value>default</value> </property> <property> <name>oozie.hive.defaults</name> <value>/user/user1/oozie/hive-site.xml</value> </property> </configuration> <script>script.q</script> <param>INPUT_TABLE=SampleTable</param> <param>OUTPUT=/user/user1/output-data/hive</param> </hive> <ok to="end"/> <error to="fail"/> </action> <kill name="fail"> <message>Hive failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end name="end"/> </workflow-app>

So here hive-site.xml is the xml site present in the $ HIVE_HOME / conf folder.
The script.q file contains the actual bush request. select * from ${INPUT_TABLE} .

how and where can we use the OUTPUT parameter?

+6
source

Source: https://habr.com/ru/post/915199/

More articles:


All Articles