Buffered Loggers (Java)

Question

I love the idea of ​​creating a logging system that pushes log operators into an internal buffer until it reaches a given capacity, and then flushes (flushes) the entire buffer at once.

This is because I like to sprinkle a lot of TRACE statements on all of my methods (so that I can see what happens on each line, it makes debugging easier, at least for me). And I'm afraid that with the (potentially) hundreds / thousands of journal operators shooting everywhere that such a large amount of I / O will swamp my programs.

A “buffered” registrar solution can facilitate this.

Three questions:

• Does something like this already exist? I hate to reinvent the wheel here, but online searches have not returned anything.
• I thought about the fact that I can very much lose log entries whenever the program stops unexpectedly (runtime exceptions, etc.), and the log was not reset. In this case, I would like the registrar to override finalize () so that if the program terminates with elements still in its buffer, it can clear (publish) them before exiting. Thoughts?
• Is this a terrible idea? If so, why!

Answer 1

Do not reinvent this wheel if it can be avoided. Take a look at Log4j or better slf4j .

Log4j and slf4j are very effective if you are not tracking, so on a production system you can turn off the logging level and still have good performance.

Both log4j and slf4j are immediately written to the log files and hidden, do not perform buffering by default, for the very good reason that you want to see an exception in the log file that caused your failure. If you really want to add buffering, you can do it ( FileAppender # bufferedIO )

As for finalize (), it cannot be called on exit. From System # runFinalizersOnExit .

Outdated. This method is inherently unsafe. This can cause finalizers to invoke live objects, while other threads are simultaneously manipulating these objects, leading to unstable behavior or a deadlock. Enable or disable finalization on exit; to do this indicates that finalizers of all objects that have finalizers that have not yet been automatically called must be run before Java run time. By default, exit finalization is disabled .

My emphasis. Thus, no, it seems that a buffered logger would have inherent problems.

Answer 2

I will confirm the third question you asked. This is a terrible idea, and it has nothing to do with your programming skills. It relates to the number of scenarios that projects are exposed to, such as log4j, and to the various edge cases that they took into account. I think this event is large enough for you and will cause a lot of headaches in the future.

Answer 3

Log4J already supports I / O buffering. Some useful Log4J performance tips are here.

Answer 4

I do not know the existing frameworks, AFAIK there is no "buffer" in the local memory. But check out standard Log4J, etc.

finalize() is not the case. Use Runtime.addShutdownHook() . View recently what I learned .

Answer 5

We do the same with our production servers, except that they run log messages over a network connection. We simply have a limited linked queue and make decisions on the removal of debug messages and storing errors if the queue is full. Then we have a thread in the background that starts messages to the server.

• I do not know such a thing as the log4j or slf4j part.

• The only way to lose messages is if the JVM crashes, in which case the write buffer will have the same problems. If you are not writing a thread to a daemon, then it can end up writing log messages when the application leaves before exiting.

• The only reason this would be a terrible idea is not a performance issue when registering. Perhaps it would be nice to make sure that you isolate your logging calls with your own slave wrapper if you need to queue them for the future. But in the short term, just write log4j or something else. You can also put some statistics in this packaging class to see that registration messages generally support the process, and then make an informed decision about improving its performance by queues.

Answer 6

As already mentioned, do not reinvent the wheel - use log4j or even the best magazine (its successor), but do it wisely. By default, simple applications that write and hide on disk will immediately affect the overall performance of your application. Typically, appender does the hard work on the caller thread. To avoid this, you either write your own appender, which does it asynchronously (with buffering), or uses wrappers - depending on the structure, you can wrap some applications with asynchronous behavior without writing a separate line of code. I have such an appender here (see the bottom of the page), you can freely use the code or just get an idea. My appender works against TCP sockets, but the problem is the same: prevent the caller from blocking and perform a cleanup in a separate thread so that the application on top does not feel it.