On Windows, how do I open for recording a file that is already open for recording by another process?

Question

I am trying to open a log file that remains open by another process and deletes the first few lines. On Unix, I would just do os.open('/tmp/file.log', os.O_NONBLOCK) , and that would bring me closer to my goal.

Now I'm stuck on Windows, and I need to somehow rotate this log without finishing the application containing the file. Is it possible?

At first I considered opening a file descriptor in the place where the application expected the log to be, and just acts as a pipe in the file descriptor in Python, but I could not find a way to do this on Windows.

I also thought that I just move the file on a regular basis and let the application recreate the file, but since it is used by another process that does not bring much benefit.

Thought O_SHLOCK , but again, that is Unix, not Windows. So I went for the mmap file and I hope that this makes it a little more flexible, but it does not lead to anything.

 import mmap import contextlib import time with open(r'test.log', 'r+') as f: with contextlib.closing(mmap.mmap(f.fileno(), 0)) as m: while 1: line = m.readline() if len(line) > 0: print line time.sleep(0.5) 

This results in the application being unable to access the file because Python is holding it (and vice versa).

I had to think about signal.SIGHUP , but it does not exist on Windows, so back to the square.

I am stuck and I tried everything, can Python help me here or do I need to switch the language?

Answer 1

Even if the application opens the file as a shared object, Python cannot, therefore, they cannot get along in appearance.

It's not so bad:). You can (open) the file using CreateFile as specified by Augusto. You can use the standard ctypes module for this. In question Using construction as an argument to a function with the python ctypes module , you can see how to do this. Then you need to associate the C runtime file descriptor with the existing operating system file system descriptor obtained in the previous step. You can use _open_osfhandle from the MS C runtime library (CRT) for this. You can call it again using ctypes; you can access it as ctypes.cdll.msvcrt._open_osfhandle . Then you need to associate the Python file object with the existing C runtime file descriptor that you got in the previous step. To do this, in Python 3, you simply pass the file descriptor as the first argument to the open built-in function. According to the docs

file is either a string or a byte object indicating the path (absolute or relative to the current working directory) of the file to open or the integer file descriptor of the file to wrap .

In Python 2, you should use os.fdopen ; his task, according to the documents, is to

Return open file object associated with fd file descriptor

All of the above should not require the completion of such a simple thing. Hopefully this will be much easier when the CPython implementation on Windows starts using its own Windows API for files instead of going through the C runtime library, which does not allow access to many features of the Windows platform. See more about the release of the new version of io.FileIO using the native Windows API .

Answer 2

Do you have any control over the application generating the log file? Because depending on how the file is opened by this application, you really cannot change it.

This link may seem off-topic here, but deep in Windows, which determines file access for another application, this is the dwShareMode parameter of the dwShareMode function: http://msdn.microsoft.com/en-us/library/windows/desktop/aa363858% 28v = vs. 85% 29.aspx

The application should include FILE_SHARE_WRITE and possibly FILE_SHARE_DELETE , plus it should clear and update the file position every time it writes a file. Looking at the Python documentation for open() , there is no such detailed parameter.