Yarn: make yarn-site.xml changes effective on cluster work nodes

We have a sparking application running on HDFS 2.7.3 with yarn as a resource manager. When the application starts, these two folders:

  /tmp/hadoop/data/nm-local-dir/filecache
    /tmp/hadoop/data/nm-local-dir/filecache 

are filled and therefore the disk. Therefore, from my research it turned out that setting these two properties in the yarn-site.xml file would help.

<property>
            <name>yarn.nodemanager.localizer.cache.cleanup.interval-ms</name>
            <value>2000</value>
    </property>
    <property>
             <name>yarn.nodemanager.localizer.cache.target-size-mb</name>
             <value>2048</value>
    </property>

I configured them on yarn-site.xml for each namenode and masternode and restarted the yarn on masternode using stop-yarn.sh start-yarn.sh , however this does not seem to help. Am I missing something while making configuration changes? And how can we make sure that these changes apply to all work nodes?

+4

Source: https://habr.com/ru/post/1666543/


All Articles