Find the latest file in a directory without reading its contents

Question

I am trying to find the last file in a huge file system. One way to do this is to go through all the directories - one at a time, read its contents, select the last file, etc.

The obvious downside is that I have to get all the files in a specific directory. I was wondering if there is a “magic” call in Python [1] that Unix supports in order to get only the last file in a directory.

[1]. My application is in Python, but if the turnkey solution does not exist in stdlib, suggest alternatives to C (lanuage) using system calls. I am ready to write a C-extension and do the job.

thanks

update . Suppose I have to offer an explanation why a solution like inotify will not work for me. I was just looking for a system call using Python / C that could give me the latest file. Yes, it would be possible to inotify (or similar comprehensive configuration), which controls FS changes, but taking into account a random directory, how can I find the last file, the essence of the matter.

Answer 1

I do not believe that, in general, Unix or Posix systems support notification of a change in the independent file system on the platform.

However, there are many unixy systems that do:

• OS X
  
  • FSEvents
  • fslogger
  • DTrace + syscall
  • pnotify
  • Kqueue
• Linux:
  
  • inotify
  • pnotify
  • epoll
• Solaris
  • Dtrace
• BSD
  • pnotify
  • Kqueue
• Python:
  • Kqueue is directly supported in Python only on BSD
  • Pyinotify for Linux only
  • FSEvents only on Darwin / OS X
  • Gamin for Linux and BSD only

Others suggested trying to interpret ls . Do not do this. If you are forced to use the Unix tool, most Unix / Linux / Posix flavors also have stat as a utility. The stat utility has custom output, and you can set the fields you want to parse. This is part of the core GNU utilities.

Answer 2

Do you consider using pyinotify , which can view a directory and subdirectories?

This may require that your code be a stream, say, an observer stream that records the latest changes to the main stream for polling.

Alternatively, you can use popen and get the result 'ls -t | head -1 '

Answer 3

Unix does not have a portable API for this. Most file systems do not index files inside directories by their mtime (or ctime), so even if this happened, perhaps it would not be faster than doing it yourself.

Answer 4

You do not need to use Python for this task, wrap python on top of Unix utilities that better understand the file system and can provide you with this information. For instance,

Make ls -ltr |tail -1 in this directory, and the result is a string, split it and get the last item you want to find.

 import subprocess targetdir = 'foo' #ls -ltr |tail -1 list_reverse = subprocess.Popen(['ls','-t',targetdir],stdout=subprocess.PIPE) tail_call = subprocess.Popen(['head','-1'],stdin=list_reverse.stdout,stdout=subprocess.PIPE) out,err = tail_call.communicate() print out.split()[-1]