How to collapse a log file when run in a log

Question

I would like to configure logback to do the following.

• File entry
• Download the file when it reaches 50 MB.
• Keep logs for 7 days
• When starting, always create a new file (make a roll)

Everything works for me, with the exception of the last element, the launch roll. Does anyone know how to achieve this? Here config ...

<appender name="File" class="ch.qos.logback.core.rolling.RollingFileAppender"> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg \(%file:%line\)%n</Pattern> </layout> <File>server.log</File> <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>server.%d{yyyy-MM-dd}.log</FileNamePattern> <!-- keep 7 days' worth of history --> <MaxHistory>7</MaxHistory> <TimeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP"> <MaxFileSize>50MB</MaxFileSize> </TimeBasedFileNamingAndTriggeringPolicy> </rollingPolicy> </appender> 

Answer 1

None of the other suggestions were suitable for my situation. I did not want to use a solution based on sizes and time, because this requires the configuration of MaxFileSize, and we use a strictly temporary policy. Here's how I finished loading the file at startup using TimeBasedRollingPolicy:

 @NoAutoStart public class StartupTimeBasedTriggeringPolicy<E> extends DefaultTimeBasedFileNamingAndTriggeringPolicy<E> { @Override public void start() { super.start(); nextCheck = 0L; isTriggeringEvent(null, null); try { tbrp.rollover(); } catch (RolloverFailure e) { //Do nothing } } } 

The trick is to set the nextCheck value to 0L, so that isTriggeringEvent () will think it is time to flip the log file. Thus, it will execute the code necessary to calculate the file name, as well as conveniently reset the nextCheck time value. A subsequent call to rollover () causes the log file to be scanned. Since this only happens at startup, this is a better solution than those that perform comparisons inside isTriggerEvent (). No matter how small this comparison is, it nevertheless somewhat degrades performance with each message in the log. This also causes the rollover to occur immediately upon startup, instead of waiting for the first log event.

@NoAutoStart annotations are important to prevent Joran from executing the start () method before completing all other initialization. Otherwise, you will get a NullPointerException.

Here is the configuration:

  <!-- Daily rollover appender that also appends timestamp and rolls over on startup --> <appender name="startupDailyRolloverAppender" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>${LOG_FILE}</file> <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <fileNamePattern>${LOG_FILE}.%d{yyyyMMdd}_%d{HHmmss,aux}</fileNamePattern> <TimeBasedFileNamingAndTriggeringPolicy class="my.package.StartupTimeBasedTriggeringPolicy" /> </rollingPolicy> <encoder> <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern> </encoder> </appender> 

Hope this helps!

Answer 2

It works for me using the following class as timeBasedFileNamingAndTriggeringPolicy:

 import java.io.File; import java.util.concurrent.atomic.AtomicBoolean; import ch.qos.logback.core.joran.spi.NoAutoStart; import ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP; @NoAutoStart public class Trigger<E> extends SizeAndTimeBasedFNATP<E> { private final AtomicBoolean trigger = new AtomicBoolean(); public boolean isTriggeringEvent(final File activeFile, final E event) { if (trigger.compareAndSet(false, true) && activeFile.length() > 0) { String maxFileSize = getMaxFileSize(); setMaxFileSize("1"); super.isTriggeringEvent(activeFile, event); setMaxFileSize(maxFileSize); return true; } return super.isTriggeringEvent(activeFile, event); } } 

Answer 3

I found another logFile logging solution once when the application starts.

I use logback RollingFileAppender with logback FixedWindowRollingPolicy and my own implementation of TriggeringPolicy<E> .

FixedWindowRollingPolicy gets the file NamePattern for the new logFile, where %1 is the new file number. MaxIndex means the maximum amount of my "history". Additional Information: FixedWindowRollingPolicy

My implementations TriggeringPolicy returns true for first when isTriggeringEvent (...) is called. Thus, WindowRollingPolicy rolls the log files when the policy is called for the first time, and then it will not roll over again.

xml configuration for RollingFileAppender :

 <configuration> ... <appender name="FILE_APPENDER" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>logFile.log</file> <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy"> <fileNamePattern>logFile.%i.log</fileNamePattern> <minIndex>1</minIndex> <maxIndex>4</maxIndex> </rollingPolicy> <triggeringPolicy class="my.classpath.RollOncePerSessionTriggeringPolicy"/> </appender> ... </configuration> 

TriggeringPolicy :

 package my.classpath; import ch.qos.logback.core.rolling.TriggeringPolicyBase; import java.io.File; public class RollOncePerSessionTriggeringPolicy<E> extends TriggeringPolicyBase<E> { private static boolean doRolling = true; @Override public boolean isTriggeringEvent(File activeFile, E event) { // roll the first time when the event gets called if (doRolling) { doRolling = false; return true; } return false; } } 

Answer 4

Overriding the isTriggeringEvent () method in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP should work well. Just return true for the first time the TriggeringEvent () method is called.

Answer 5

To solve using existing components, logback offers uniquely named files: http://logback.qos.ch/manual/appenders.html#uniquelyNamed

During the application development phase or in the case of short-lived applications, for example. packaged applications, it is advisable to create a new log file each time a new application is launched. This is pretty easy to do with the <timestamp> element.

 <?xml version="1.0" encoding="UTF-8"?> <configuration> <timestamp key="startTimestamp" datePattern="yyyyMMddHHmmssSSS"/> <appender name="File" class="ch.qos.logback.core.rolling.RollingFileAppender"> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg \(%file:%line\)%n</Pattern> </layout> <file>server-${startTimestamp}.log</file> <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <FileNamePattern>server-${startTimestamp}-%d{yyyy-MM-dd}-%i.log</FileNamePattern> <!-- keep 7 days' worth of history --> <MaxHistory>7</MaxHistory> <TimeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP"> <MaxFileSize>1KB</MaxFileSize> </TimeBasedFileNamingAndTriggeringPolicy> </rollingPolicy> </appender> <root level="DEBUG"> <appender-ref ref="File" /> </root> </configuration> 

---

UPDATED for logback-1.2.1

 <?xml version="1.0" encoding="UTF-8"?> <configuration> <timestamp key="startTimestamp" datePattern="yyyyMMddHHmmssSSS"/> <appender name="File" class="ch.qos.logback.core.rolling.RollingFileAppender"> <layout class="ch.qos.logback.classic.PatternLayout"> <Pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg \(%file:%line\)%n</Pattern> </layout> <file>server-${startTimestamp}.log</file> <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy"> <fileNamePattern>server-${startTimestamp}-%d{yyyy-MM-dd}-%i.log</fileNamePattern> <maxFileSize>10MB</maxFileSize> <!-- keep 7 days' worth of history --> <maxHistory>7</maxHistory> <totalSizeCap>20GB</totalSizeCap> </rollingPolicy> </appender> <root level="DEBUG"> <appender-ref ref="File" /> </root> </configuration> 

Answer 6

Ceki's solution does not seem to work for me, but it seems to at least partially work.

It will explode because it cannot see the rolling policy when running TimeBasedFileNamingAndTriggeringPolicyBase . With some hacker I got it to do some logging, and with some more I got it to watch the trigger, but then it broke again because it could not solve one of the file properties ... The package is logback one so I can get into some of the internal parts to reproduce some logic in SizeAndTimeBasedFNATP#isTriggeringEvent and call computeCurrentPeriodsHighestCounterValue . I think something along these lines may work, just until you find the magic combination. I really hope that I am doing something stupid, because otherwise I would think that this would mean either opening some details for subclassing, or including this line in the journal as another transition / start policy.

logback.xml: performed various orderings triggeringPolicy , TimeBasedFileNamingAndTriggeringPolicy inside and outside rollingPolicy .

 <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>${LOG_DIR}/${LOG_FILE_BASE}.log</file> <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <fileNamePattern>${LOG_DIR}/${LOG_FILE_BASE}.%d{yyyy-MM-dd}.%i.log</fileNamePattern> <MaxHistory>7</MaxHistory> <TimeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.RollOnStartupPolicy" /> </rollingPolicy> <filter class="ch.qos.logback.classic.filter.ThresholdFilter"> <level>INFO</level> </filter> <encoder> <pattern>%msg%n</pattern> </encoder> </appender> 

Launch Policy:

 package ch.qos.logback.core.rolling; public class RollOnStartupPolicy<E> extends SizeAndTimeBasedFNATP<E> { private final AtomicBoolean firstTime = new AtomicBoolean(true); @Override public boolean isTriggeringEvent(File activeFile, E event) { if (!firstTime.get()) { // fast path return false; } if (firstTime.getAndSet(false)) { return true; } return false; } } 

An exception:

 java.lang.NullPointerException at at ch.qos.logback.core.rolling.TimeBasedFileNamingAndTriggeringPolicyBase.start(TimeBasedFileNamingAndTriggeringPolicyBase.java:46) at at ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP.start(SizeAndTimeBasedFNATP.java:36) at at ch.qos.logback.core.joran... [snip joran config] 

Answer 7

Create your own subclass of ch.qos.logback.core.rolling.TimeBasedRollingPolicy and override it start

 public class MyPolicy extends ch.qos.logback.core.rolling.TimeBasedRollingPolicy { public void start ( ) { super.start( ); rollover( ); } } 

Answer 8

I have the following to work (combining ideas from previous answers). Note. I dealt with files based on size, not time, but I guess the same solution works.

 public class StartupSizeBasedTriggeringPolicy<E> extends ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy<E> { private final AtomicReference<Boolean> isFirstTime = new AtomicReference<Boolean>(true); @Override public boolean isTriggeringEvent(final File activeFile, final E event) { //this method appears to have side-effects so always call boolean result = super.isTriggeringEvent(activeFile, event); return isFirstTime.compareAndSet(true, false) || result; } 

}

Answer 9

I finally figured it out. I can roll in size, time and launch. Here is the solution:

Create your own class

 @NoAutoStart public class StartupSizeTimeBasedTriggeringPolicy<E> extends SizeAndTimeBasedFNATP<E> { private boolean started = false; @Override public boolean isTriggeringEvent( File activeFile, E event ) { if ( !started ) { nextCheck = 0L; return started = true; } return super.isTriggeringEvent( activeFile, event ); }; } 

2nd configure logback

 <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>${LOGS_DIR}/${FILE_NAME}.log</file> <encoder> <pattern>%d [%thread] %-5level %logger{50} - %msg%n</pattern> </encoder> <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <fileNamePattern>${LOGS_DIR}/${FILE_NAME}.%d{yyyy-MM-dd}_%d{HHmmss,aux}.%i.log.zip</fileNamePattern> <maxHistory>30</maxHistory> <TimeBasedFileNamingAndTriggeringPolicy class="my.StartupSizeTimeBasedTriggeringPolicy"> <MaxFileSize>250MB</MaxFileSize> </TimeBasedFileNamingAndTriggeringPolicy> </rollingPolicy> </appender> 

Answer 10

This solution really works, thanks a lot. However, there is one annoying glitch: when you run the program for the first time, the log starts right after its creation, when it is empty or almost empty. Therefore, I propose a fix: check if the log file exists and is not empty at the time the method is called. In addition, another cosmetic fix: rename the "initial" variable, as it hides an inherited element with the same name.

 @NoAutoStart public class StartupSizeTimeBasedTriggeringPolicy<E> extends SizeAndTimeBasedFNATP<E> { private boolean policyStarted; @Override public boolean isTriggeringEvent(File activeFile, E event) { if (!policyStarted) { policyStarted = true; if (activeFile.exists() && activeFile.length() > 0) { nextCheck = 0L; return true; } } return super.isTriggeringEvent(activeFile, event); } } 

In addition, I believe that it works correctly with the log version of version 1.1.4-SNAPSHOT (I received the source code and compiled it myself), but it does not fully work with the release of version 1.1.3. In 1.1.3, it correctly defines files with the specified time zone, but the rollover still occurs at midnight by default time zone.

Answer 11

The API has changed (e.g. setMaxFileSize no longer exists) and many of the above functions don't seem to work, but I have something that works for me against logback 1.1.8 (the latter currently).

I wanted to start at startup and rolls in size, but not the time. It does:

 public class RollOnStartupAndSizeTriggeringPolicy<E> extends SizeBasedTriggeringPolicy<E> { private final AtomicBoolean firstTime = new AtomicBoolean(); public boolean isTriggeringEvent(final File activeFile, final E event) { if (firstTime.compareAndSet(false, true) && activeFile != null && activeFile.length() > 0) { return true; } return super.isTriggeringEvent(activeFile, event); } } 

With this, you also need a rolling policy. FixedWindowRollingPolicy will probably do, but I do not like it because I want to store a large number of files, and for this it is very inefficient. Something that the number is gradually increasing (instead of sliding like FixedWindow) will work, but this does not exist. While I write myself, I decided to use time instead of counting. I would like to extend the current registration code, but for time-based material, driving and launch policies are often combined into one class, and there are nesting and circular logs and fields without getters, so I found this pretty impossible. So I had to do a lot from scratch. I support it simply and do not implement features such as compression - I would like to use them, but I'm just trying to keep it simple.

 public class TimestampRollingPolicy<E> extends RollingPolicyBase { private final RenameUtil renameUtil = new RenameUtil(); private String activeFileName; private String fileNamePatternStr; private FileNamePattern fileNamePattern; @Override public void start() { super.start(); renameUtil.setContext(this.context); activeFileName = getParentsRawFileProperty(); if (activeFileName == null || activeFileName.isEmpty()) { addError("No file set on appender"); } if (fileNamePatternStr == null || fileNamePatternStr.isEmpty()) { addError("fileNamePattern not set"); fileNamePattern = null; } else { fileNamePattern = new FileNamePattern(fileNamePatternStr, this.context); } addInfo("Will use the pattern " + fileNamePattern + " to archive files"); } @Override public void rollover() throws RolloverFailure { File f = new File(activeFileName); if (!f.exists()) { return; } if (f.length() <= 0) { return; } try { String archiveFileName = fileNamePattern.convert(new Date(f.lastModified())); renameUtil.rename(activeFileName, archiveFileName); } catch (RolloverFailure e) { throw e; } catch (Exception e) { throw new RolloverFailure(e.toString(), e); } } @Override public String getActiveFileName() { return activeFileName; } public void setFileNamePattern(String fnp) { fileNamePatternStr = fnp; } } 

And then the config looks like

 <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> <encoder> <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern> </encoder> <file>/tmp/monitor.log</file> <rollingPolicy class="my.log.TimestampRollingPolicy"> <fileNamePattern>/tmp/monitor.%d{yyyyMMdd-HHmmss}.log</fileNamePattern> </rollingPolicy> <triggeringPolicy class="my.log.RollOnStartupAndSizeTriggeringPolicy"> <maxFileSize>1gb</maxFileSize> </triggeringPolicy> </appender> 

If you are upset, this is not decided initially, vote for him at

http://jira.qos.ch/browse/LOGBACK-204

http://jira.qos.ch/browse/LOGBACK-215

(these were years, and for me this is absolutely important functionality, although I know that many other frameworks also fail)