Implement a Semi-Round-Robin file that you can deploy and save on demand

Ok, this headline will be a bit confusing. Let me try to explain this a little better. I am creating a logging program. The program will have 3 main states:

  • Write to the circular buffer file, saving only the last 10 minutes of data.

  • Write to the spooled file, ignoring the time (write all the data).

  • Rename the entire buffer file and run a new one in the last 10 minutes of data (and change the state to 1).

Now this is a usage example. From time to time, I experience some network bottlenecks in our network. Therefore, I want to create a system to record TCP traffic when it detects a bottleneck (discovery through Nagios). However, by the time bottlenecks were discovered, most of the useful data had already been transferred.

So, I would like deamon to have something like this all the time dumpcap. In normal mode, it will only store the last 10 minutes of data (since it makes no sense to keep the load on the boat if it is not needed). But when Nagios warns, I will send a signal to the deamon to store everything. Then, when Nigos recovers, it will send another signal to stop saving and flush the buffer to the save file.

Now the problem is that I do not see how to accurately store rotating 10 minutes of data. I can store a new file every 10 minutes and delete the old ones if in mode 1. But it seems a little dirty to me (especially when it comes to figuring out when a warning occurred in the file).

, , , 10:00 . 10 , , , "" .

? 1 ( )? , - -?

, ( Python, - . , )...

+3
3

, , MINUTES+1 ( 11) . .

/ 10 , " ", , .

" - ", , , - :)

, , - X . 0 .

+3

, , , MongoDB Capped Collections - , , , .

, FIFO- ( ). "RRD", . , , , ; , .

, , , 10 . Nagios , , , . MongoDB , , 10- .

+1

What is the advantage of using only the last 10 minutes of magazines? to implement this, you will need to constantly check the old logs and delete them from the file, and then overwrite the file. such functionality can be more easily achieved using some database, for example. SQLite

log timestamps give you the same and much more. Just save the two log files as described, if the log file already has 10 minutes of the log, rename it (overwrite the older one) and start writing to a new file.

0
source

Source: https://habr.com/ru/post/1790486/


All Articles