Deployment mode in "SPARK-SUBMIT"

Question

In SPARK-SUBMIT, what is the difference between the modes “yarn”, “yarn-cluster”, “yarn-client”?

./bin/spark-submit \
  --class org.apache.spark.examples.SparkPi \
  --master yarn-cluster \  # can also be `yarn-client` for client mode
  --executor-memory 20G \
  --num-executors 50 \
  /path/to/examples.jar \
  1000

https://spark.apache.org/docs/1.1.0/submitting-applications.html

Answer 1

For Spark on YARN, you can specify either a client thread or a yarn cluster. Yarn-client runs the driver program in the same JVM as the original scheme, while the yarn cluster runs the Spark driver in one of the NodeManager containers.

: https://spark.apache.org/docs/1.1.0/running-on-yarn.html , Spark YARN. Spark , YARN , . "-" , YARN.