How to disable apache zookeeper debugging messages (AWS EMR)?

Question

How to disable DEBUG messages on AWS Elastic MapReduce Master node?

hbase(main):003:0> list TABLE mydb 1 row(s) in 0.0510 seconds hbase(main):004:0> 00:25:17.104 [main-SendThread(ip-172-31-14-206.ec2.internal:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for sessionid: 0x1493a5c3b78001b after 1ms hbase(main):005:0* 00:26:17.165 [main-SendThread(ip-172-31-14-206.ec2.internal:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for sessionid: 0x1493a5c3b78001b after 1ms 

Answer 1

I do not know if there is any parameter in the web interface to change the verbosity of the journal. But there should be a Zookeeper configuration folder (which uses Log4j) $ {ZOOKEEPER_HOME} / conf , where you can edit the logj4.properties file and specify:

 zookeeper.console.threshold=WARN 

But I think that these changes should not be done directly in Zookeeper, but in the HBase log configuration - $ {HBASE_HOME} / conf (HBase also uses Log4j), because HBase can control Zookeeper. There are several options that you can edit there:

 # Define some default values that can be overridden by system properties hbase.root.logger=INFO,console hbase.security.logger=INFO,console # Main log level log4j.threshold=ALL # Zookeeper log level log4j.logger.org.apache.zookeeper=INFO 

To find this file, you can try the following command:

 $ find /* -name "log4j.properties" | grep -E "zookeeper|hbase" /hadoop/zookeeper/conf/log4j.properties /hadoop/hbase/conf/log4j.properties 

Answer 2

The accepted answer to this question helps suppress debug log messages not only for the hbase shell , but also for all other hbase daemons (for example, region server, zookeeper). All you need to do is add:

 <configuration> <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender"> <encoder> <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern> </encoder> </appender> <root level="error"> <appender-ref ref="STDOUT" /> </root> </configuration> 

in ~/hbase/conf/logback.xml and reload all services or hbase shell.