All geek questions in one place

Setting oozie workflow properties for HA JobTracker

With an Oozie workflow, you must specify the JobTracker cluster in the workflow properties. This is easy when you have one JobTracker:

JobTracker = hostname: port

When the cluster is configured to work with HA (high availability) JobTracker, I need to configure my property files to be able to delete any of the JobTracker hosts without updating all my property files when JobTracker has failed to switch to the second node.

When accessing one JobTracker via http, it redirects to another if it is not running, but oozie does not use http, so there is no redirection, which leads to a workflow failure if the properties file specifies a task that is not running.

How can I configure my properties file to handle JobTracker running in HA?

+4
source share
2 answers

I just finished setting up some Oozie workflows to use HA JobTrackers and NameNodes. The key should use the logical name of the configured HA service, and not the individual host names or ports. For example, the default name of HA JobTracker is "logicaljt". Replace the host name: port with "logicaljt", and everything should work only until the node from which you are running Oozie has correctly configured the appropriate hdfs site and sitemaps (implicitly because it is part of cluster or explicitly due to the addition of a gateway role to it).

+2
source

Please specify a name service for the cluster in which HA is enabled. eg:

in the properties file

namenode=hdfs://<nameserivce> jobTracker=<nameservice>:8032
0
source

Source: https://habr.com/ru/post/1487321/

More articles:


All Articles