Tail sampling logs

Question

I have a process that needs to do periodic processing on an ever-growing log file. Right now, how I do this is pretty simple (I will include a bash script if you're really interested).

• Run tail -n0 -f $ FILE
• Each iteration:
  • Kill the tail
  • Move old sample
  • Launch a new tail

This solves the problem of not overlapping, but I'm worried about 1 or 2 lines that I can skip. Is there a better way to do this to avoid overlapping (and “below” the circle)?

Answer 1

"Move the old pattern", I assume that you mean the rotation of the file, moving the current and replacing it with a new file.

If so, you can use the --follow=name option for tail instead of -f . This follows the file by name, not the file descriptor, which allows it to continue even if the files are replaced. Then you can leave your tail running when replacing files and not skip any entries.

For a more robust approach, also include --retry or just use -f , which implies --follow=name --retry .

On the man page:

-f, --follow[={name|descriptor}]

display added data as the file grows; -f , --follow and --follow=descriptor equivalent

--retry

keep trying to open the file even if it is not available when the tail starts or later becomes inaccessible - only useful with -f

-f

same as --follow=name --retry