How to define memory separately for hadoop daemons in hadoop-env.sh

Question

As I know, hasoop-env.sh is a configuration file for settings for hadoop daemons. In this file, as I can determine HADOOP_HEAPSIZE , it is a property to determine the heap size for daemons. This means that it will work for all daemons, such as namenode, datanode, task tracker, job tracker and secondary namenode, for each machine 1000 MB of memory is required. If so, how can I do different for everyone.

I look at the following few entries in the hadoop-env.sh file

  export HADOOP_NAMENODE_OPTS = "- Dcom.sun.management.jmxremote $ HADOOP_NAMENODE_OPTS"
 export HADOOP_SECONDARYNAMENODE_OPTS = "- Dcom.sun.management.jmxremote $ HADOOP_SECONDARYNAMENODE_OPTS"
 export HADOOP_DATANODE_OPTS = "- Dcom.sun.management.jmxremote $ HADOOP_DATANODE_OPTS"
 export HADOOP_BALANCER_OPTS = "- Dcom.sun.management.jmxremote $ HADOOP_BALANCER_OPTS"
 export HADOOP_JOBTRACKER_OPTS = "- Dcom.sun.management.jmxremote $ HADOOP_JOBTRACKER_OPTS"

Are these entries for a separate memory allocation for each daemon, if so, what is -Dcom.sub.management.jmxremote, as far as I know, the syntax for allocating java heap memory is similar to -Xmx <size> m .

This is also where the task tracker is. and what is a balancer.

Answer 1

-Dcom.sun.management.jmxremote is used to access the JMX client. http://docs.oracle.com/javase/7/docs/technotes/guides/management/agent.html

Yes, the parameters listed in your question are used as JVM parameters when starting these daemons respectively. Thus, HADOOP_NAMENODE_OPTS is used when starting the node name, HADOOP_DATANODE_OPTS is used when starting the node daemon, etc. You can add the -Xmx option to these variables with different values ​​so that different types of daemons have different heap sizes.