How to schedule sqoop action using oozie

Question

I'm new to Oozie, just wondering - How do I schedule sqoop to work with Oozie. I know that the sqoop action can be added as part of the Oozie workflow. But how can I schedule sqoop action and run it like every 2 minutes or 8 pm every day automatically (just lay the cron job)?

Answer 1

You need to create a coordinator.xml file with start, end and frequency. Here is an example

<coordinator-app name="example-coord" xmlns="uri:oozie:coordinator:0.2" frequency="${coord:days(7)}" start="${start}" end= "${end}" timezone="America/New_York"> <controls> <timeout>5</timeout> </controls> <action> <workflow> <app-path>${wf_application_path}</app-path> </workflow> </action> </coordinator-app> 

Then create a coordinator.properties file similar to this:

 host=namenode01 nameNode=hdfs://${host}:8020 wf_application_path=${nameNode}/oozie/deployments/example oozie.coord.application.path=${wf_application_path} start=2013-07-13T07:00Z end=2013-09-31T23:59Z 

Upload the Coordinator.xml file to hdfs and then submit your coordinator work with something like

 oozie job -config coordinator.properties -run 

Check out the documentation http://oozie.apache.org/docs/3.3.2/CoordinatorFunctionalSpec.html for some examples.

Answer 2

I think the next blog will be very helpful.

http://www.tanzirmusabbir.com/2013/05/chunk-data-import-incremental-import-in.html