Is the spark.yarn.containerLauncherMaxThreads property in the Spark Yarn configuration affect the overall runtime?

The documentation for the claim spark.yarn.containerLauncherMaxThreadsexplains the property here as

The maximum number of threads to use in the YARN Application Wizard to run artist containers.

Does this mean that only 25 performers will be performed in parallel?

Is it possible to increase performance by tuning this property?

+4

yarn apache-spark

Vishnu Jun 03 '17 at 6:24

No one has answered this question yet.

See related questions:

192

What are the workers, executors, cores in the Spark Standalone cluster?

25

What is the yarn-client mode in Spark?

12

Spark Launch Log runs on YARN

8

Performance issues for spark on YARN

4

Resources / Documentation on how the fault tolerance process works for Spark Driver (and its YARN container) in thread cluster mode

1

How does dynamic allocation of spark resources work in YARN (in relation to NodeManagers)?

1

Spark increases the number of performers in yarn mode

0

Spark streaming application configuration with YARN

0

How to set up a cluster of yarn using a spark?

0

How to configure yarn in Cloudera to run Spark artists on all nodes of the cluster?

Source: https://habr.com/ru/post/1678449/

All Articles