Replace default application.conf file with spark-submit

My code works like:

val config = ConfigFactory.load

By default, key-value pairs are received from application.conf . Then I use -Dconfig.file = to point to another conf file.

It works great for the command below:

dse -u cassandra -p cassandra spark-submit 
    --class packagename.classname --driver-java-options 
    -Dconfig.file=/home/userconfig.conf /home/user-jar-with-dependencies.jar

But now I need to split userconfig.conf into 2 files. I tried the command below. This does not work.

dse -u cassandra -p cassandra spark-submit 
    --class packagename.classname --driver-java-options 
    -Dconfig.file=/home/userconfig.conf,env.conf 
     /home/user-jar-with-dependencies.jar
+4
source share
1 answer

defaults.conf, : 1) , 'properties-file' 2), keu --conf 3), , sparkConf

akka application.conf?

+1

Source: https://habr.com/ru/post/1626301/


All Articles