Create an individual account number instance file

Question

I have a list of account numbers separated by a string generated by a transaction log.

How to consolidate this list into a file that will have only one occurrence of each account, and not many (where more than one transaction was recorded per account)?

Python is preferred, but I can also use C.

Answer 1

with open(filename) as fin, open(newfilename, 'w') as fout: fout.writelines(set(fin)) 

Answer 2

I don't know what your log file looks like, but this should work well

in python:

 file = open('filename.txt', r+) accountNos = set(file) file.truncate() for x in accountNos: file.write(x) file.close() 

This prints each line from the file and stores them in a set. A collection is a data structure that stores only unique elements and removes duplicates. In the second loop, you write the contents of this set back to the file.

Answer 3

It would be helpful if you included a sample log file and your operating system.

If you are in a UNIX environment, it is very simple using awk and sort.

If your log file (called say log.txt) contains account information as the third word on each line (see the example log file below):

 LOG WARNING 12345 cancelled .... LOG WARNING 67482 subscribed .... 

See the dollar sign below, which is the command line:

 $ awk '{print $3}' log.txt | sort -u 

If you are in a Windows environment, you can download cygwin ( http://www.cygwin.com/ ) for Windows, install it and run the above command from the command line.

Answer 4

If order is important, you can use collections.OrderedDict :

 from collections import OrderedDict with open('input') as fin, open('output', 'w') as fout: uniques = OrderedDict.fromkeys(fin) fout.writelines(uniques)