If the file is very large and you only perform rewriting operations (without inserts or exceptions), the mmap module allows you to treat the file as essentially a large mutable string. This allows you to edit the contents of the file by byte or to edit entire fragments without loading them completely into memory (the mmap object will lazily load part of the file into the desired and desired memory).
It is a little cumbersome to use, but if necessary it is extremely powerful.
Example:
$ xxd data 0000000: a15e a0fb 4455 1d0f b104 1506 0e88 08d6 .^..DU.......... 0000010: 8795 d6da 790d aafe 9d6a 2ce5 f7c3 7c97 ....y....j,...|. 0000020: 4999 ab6b c728 352e b1fd 88e0 6acf 4e7d I..k.(5.....jN} $ python >>> import mmap >>> f = open('data', 'a+') >>> m = mmap.mmap(f.fileno(), 0) >>> m[24:48] '\x9dj,\xe5\xf7\xc3|\x97I\x99\xabk\xc7(5.\xb1\xfd\x88\xe0j\xcfN}' >>> m[24:48] = 'a'*24 >>> m.close() >>> f.close() >>> ^D $ xxd data 0000000: a15e a0fb 4455 1d0f b104 1506 0e88 08d6 .^..DU.......... 0000010: 8795 d6da 790d aafe 6161 6161 6161 6161 ....y...aaaaaaaa 0000020: 6161 6161 6161 6161 6161 6161 6161 6161 aaaaaaaaaaaaaaaa
source share