"tail -f" makes the disk full?

Question

our application server (sunOS) always gets a full disk. and our Infrastructure team stated that it was caused by too many tail -f processes. Since the application often changes the log file, did this cause a dead link and lack of disk space? I have never heard of this before. does this command really cause a full disk?

Answer 1

The space occupied by the file cannot be restored until all links to this file disappear. Therefore, any process with an open file will prevent the deletion of the file from disk.

Active tail -f following the file, for example.

If these files need to be deleted on free disk space (for example, because they are very large or there are a lot of them), then the processes surrounding these links will prevent them from being deleted and eventually lead to the disk becoming full.

Change in response to a comment on another answer:

The diagnosis you are reporting is exactly what you expect to see in the situation that Adam and I describe. df reports that a 56G disk is in use, and du reports that only 10G visible in the folder. The discrepancy is due to the fact that 46G files are standing that were deleted from the folder, but cannot be physically deleted from the disk, because some processes contain links to them.

It is enough to just experiment with this: find a file system in which you can play safely and create a huge file. Write a C program that opens the file and goes into an endless loop. Now do the following:

• Run the program
• Check df output
• rm file
• Check the df output again
• Stop your program
• Check the df output again

You will see that the result of df does not change after the rm file, but changes after the program stops (thus removing the last link to the file).

If you need even more evidence that this is happening, you can get information from the /proc file system, if you have it. In particular, find the PID of one of the tail -f processes (or other processes that you think might be the reason), and look at the /proc/<pid>/fd directory to see all the files that it opened.

(I do not have * nix at home, so I cannot verify that you will see /proc/<pid>/fd in this situation)

Answer 2

tail is a command to view the end of a file, and -f does this in real time, updating the display whenever the file itself changes. It allows you to view log files in real time.

tail can cause problems in two ways:

• If tail -f is not used correctly to write to a file instead of the interactive console, this is an inefficient means of copying the file and creating duplicate logs.
• tail -f keeps the log file active, so maintenance tasks that attempt to automatically delete the log files fail. This discards the rotation of the log file, not allowing the old age.

If your infrastructure team complains about accessing files for less than a week, you really need more disk space and / or a less sophisticated logging strategy, since they cannot support enough logs in case something goes wrong, and it’s necessary keep track of. If the logs are older than this, they may have a good point, and excessive use of tail - like anything else that supports opening a file, can prevent it from being deleted on time.

The command itself is unlikely to fill the disk, but this may be to prevent disk hygiene operations.