Select data columns from .txt to .csv

Question

I am new to python (better than I used only last week). My task seems pretty simple, but I'm struggling. I have several large text files, each of which contains many columns of data from different regions. I would like to take data from one text file and extract only the data columns that I need and write them to a new CSV file. They are currently split into a tab, but I would like the result to be separated by commas.

I have:

#YY  MM DD hh mm WVHT  SwH  SwP  WWH  WWP SwD WWD   MWD
#yr  mo dy hr mn    m    m  sec    m  sec  -  degT  degT
2010 07 16 17 00  0.5  0.5  5.0  0.3  4.0 SSE SSE   163
2010 07 16 16 00  0.6  0.5  5.9  0.3  3.8 SSE SSE   165
2010 07 16 15 00  0.5  0.5  6.7  0.3  3.6 SSE  SW   151
2010 07 16 14 00  0.6  0.5  5.6  0.3  3.8 SSE SSE   153

I only want to save: DD, WVHT and MWD

Thanks in advance, Harper

Answer 1

You need to format this question a little legibly. :)

csv python csv : http://docs.python.org/library/csv.html

EDIT: , , + csv:

import csv

csv_out = csv.writer(open('out.csv', 'w'), delimiter=',')

f = open('myfile.txt')
for line in f:
  vals = line.split('\t')
  # DD, WVHT, MWD
  csv_out.writerow(vals[2], vals[5], vals[12])
f.close()

Answer 2

- csv .

CSVReader CSVWriter:

>>> import csv
>>> csv_in = csv.reader(open('eggs.txt', 'rb'), delimiter='\t')
>>> csv_out = csv.writer(open('spam.csv', 'w'), delimiter=',')

csv.

>>> for line in csv_in:
...    csv_out.writerow(line[2], line[5], line[-1])

Answer 3

, :

2010 07 16 17 00 0,5 0,5 5,0 0,3 4,0 SSE SSE 163 2010 07 16 16 00 0,6 0,5 5,9 0,3 3,8 SSE SSE 165 2010 07 16 15 00 0,5 0,5 6,7 0,3 3,6 SSE SW 151 2010 07 16 14 00 0,6 0,5 5,6 0,3 3,8 SSE SSE 153

, . , , 2010 :

f = open('data.txt')
for line in f:
    for portion in line.split(' 2010') #space is significant
    # write to csv

If your data spans several years, then the Python module itertoolscan be very convenient. I often use a recipe grouper.

import csv
from itertools import izip_longest

csv_writer = csv.writer(open('eggs.csv', 'wb'), delimiter=',')

def grouper(n, iterable, fillvalue=None):
  """
  >>> grouper(3, 'ABCDEFG', 'x')
  ['ABC', 'DEF', 'Gxx']
  """
  args = [iter(iterable)] * n
  return izip_longest(fillvalue=fillvalue, *args)

f = open('spam.txt')
for line in grouper(22, f.split('\t')): 
    csv_writer.writerow(line[2], line[12])

Answer 4

Here is the basic thing, since this is a basic need, and since there is no widespread use of csv, here is a fragment without a csv module.

DD = 2
WVHT = 5
MWD = 12
INPUT = "input.txt"
OUTPUT = "output.csv"

from os import linesep

def main():
    t = []
    fi = open(INPUT)
    fo = open(OUTPUT, "w")
    try:
        for line in fi.xreadlines():
            line = line.split()
            t.append("%s,%s,%s" %(line[DD], line[WVHT], line[MWD]))
        fo.writelines(linesep.join(t))
    finally:
        fi.close()
        fo.close()

if __name__ == "__main__":
    main()