Python file editing

Question

I wrote a small python program as a personal utility to help me with some refactoring. It is similar to replacing unix, except that it supports regular expressions and works with all files in a directory and (optionally) all subdirectories.

The problem is that I am not replacing the place. I open the files, transferring the contents to memory, and then overwrite the file like this:

file = open('path/to/file','r')
in_string = file.read()
file.close()
# ...
#Processing logic
# ...
file = open('path/to/file','w')
file.write(out_string)
file.close()

Besides the obvious performance and memory issues that are legal, but not so much a problem for my use, but also another disadvantage of this method. SVN is worried. I can make some copies and pastes of workarounds after the fact to fix the checksum error that svn throws the commit but makes this utility pointless.

? , , . ?

Answer 1

, , . Subversion , - , .

, .svn? .svn/text-base Subversion .svn-base, , !

Answer 2

"SVN freaks out"?

, vi/emacs/etc :

f = open("/path/to/.file.tmp", "w")
f.write(out_string)
f.close()
os.rename("/path/to/.file.tmp", "/path/to/file")

(, "fsync"... , Python)

, , , , ... "" , ( 100% ) ( 100% ) - .

Answer 3

, fileinput /:

:

import fileinput

for line in fileinput.input("test.txt", inplace=1):
    print "%d: %s" % (fileinput.filelineno(), line),

Answer 4

, ? , , , " ", , vi (1).

Answer 5

' = (' // ',' w + ')'. , , .

Answer 6

, .svn, , SVN, , .

( ), , . :

• "filename.new", .
• "filename.new" "filename"

Thus, you do not risk losing data if they were killed at the wrong point. Note that the fileinput module will handle most of this for you. It can be given a sequence of files for processing, and if you specify inplace = True, redirect stdout to the appropriate file (saving a backup). Then you can structure the code as follows:

import os
import fileinput

def allfiles(dir, ignore_dirs=set(['.svn'])):
    """Generator yielding all writable filenames below dir.
    Ignores directories specified 
    """
    for basedir, dirs, files in os.walk(dir):
        if basedir in ignore_dirs:
            dirs[:]=[] # Don't recurse
            continue  # Skip this directory

        for filename in files:
            filename = os.path.join(basedir, filename)
            # Check the file is writable
            if os.access(filename, os.W_OK):
                yield filename


for line in fileinput.input(allfiles(PATH_TO_PROCESS), inplace=True):
    line = perform_some_substitution(line)
    print line.rstrip("\n") # Print adds a newline, but line already has one