I have a storm topology, and I would like to log certain events that occur in the topology into a separate log file. I am trying to create a custom appender in storm / logback / cluster.xml that will be used to log these events. Here is my cluster.xml fragment that installs everything:
<appender name="A2" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>${storm.home}/logs/custom-logger/cl-log.log</file> <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy"> <fileNamePattern>${storm.home}/logs/${logfile.name}.%i</fileNamePattern> <minIndex>1</minIndex> <maxIndex>9</maxIndex> </rollingPolicy> <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy"> <maxFileSize>100MB</maxFileSize> </triggeringPolicy> <encoder> <pattern>%d{yyyy-MM-dd_HH:mm:ss} %c{1} [%p] %m$n</pattern> </encoder> </appender> <logger name="custom-logger" additivity="false"> <level value="INFO" /> <appender-ref ref="A2"/> </logger>
I use the basic WordCountTopology to just check the example. Here is a snippet of code in which I am trying to write to a log file
public static class WordCount extends BaseBasicBolt { private static final org.slf4j.Logger CUSTOM_LOGGER = LoggerFactory.getLogger("custom-logger"); Map<String, Integer> counts = new HashMap<String, Integer>(); public void execute(Tuple tuple, BasicOutputCollector collector) { String word = tuple.getString(0); Integer count = counts.get(word); if (count == null) count = 0; count++; counts.put(word, count); CUSTOM_LOGGER.info("Emitting word[" + word + "] count["+ count + "]"); collector.emit(new Values(word, count)); } public void declareOutputFields(OutputFieldsDeclarer declarer) { declarer.declare(new Fields("word", "count")); } }
No matter what I do, I can not get these logs to appear in the user log cl-log.log file. Is it even possible in Storm to log specific events in a specific file using a log? Any help would be greatly appreciated.
source share