All geek questions in one place

How to get rid of the message "Use the default log4j message. Spark default: org / apache / spark / log4j-defaults.properties"?

I'm trying to suppress a message

Using Spark default log4j profile: org/apache/spark/log4j-defaults.properties

when I launch the Spark app. I redirected INFO messages successfully, however this message continues to be displayed. Any ideas would be greatly appreciated.

+8
source share
3 answers

Ok, so I figured out how to do this. So basically I had my own log4j.xml that was used, and therefore we saw this property. When I had my own log4j.properties file, this message went away.

+5
source

Even simpler you just cd SPARK_HOME/conf , then mv log4j.properties.template log4j.properties , then open log4j.properties and change all INFO to ERROR . Here SPARK_HOME is the root directory of your spark installation.

Some may use hdfs as their internal Spark repository and find the registration messages that hdfs actually generate. To change this, go to the HADOOP_HOME/etc/hadoop/log4j.properties . Just change hadoop.root.logger=INFO,console to hadoop.root.logger=ERROR,console . Once again, HADOOP_HOME is the root of your hadoop installation for me, it was /usr/local/hadoop .

+10
source

If you put the log4j.properties file under main/resources and test/resources , this will also happen. In this case, deleting the file from test/resources and using only the file from main/resources fixes the problem.

+3
source

Source: https://habr.com/ru/post/989362/

More articles:


All Articles