Convert binary input to text mode

Question

I am trying to read a compressed CSV file with bzip2 in Python 3.2. For an uncompressed CSV file, this works:

datafile = open('./file.csv', mode='rt')
data = csv.reader(datafile)
for e in data:    # works
    process(e)

The problem is that it only supports binary stream creation , and in Python 3 it only accepts text streams , (The same problem occurs with gzip and zip files.) BZ2Filecsv.reader

datafile = bz2.BZ2File('./file.csv.bz2', mode='r')
data = csv.reader(datafile)
for e in data:    # error
    process(e)

In particular, the specified string throws an exception _csv.Error: iterator should return strings, not bytes (did you open the file in text mode?).

I also tried data = csv.reader(codecs.EncodedFile(datafile, 'utf8')), but this does not fix the error.

How can I wrap a binary input stream so that it can be used in text mode?

Answer 1

This works for me:

import codecs, csv
f = codecs.open("file.csv", "r", "utf-8")
g = csv.reader(f)
for e in g:
    print(e)

In the case of BZ2:

import codecs, csv, bz2
f = bz2.BZ2File("./file.csv.bz2", mode="r")
c = codecs.iterdecode(f, "utf-8")
g = csv.reader(c)
for e in g:
    print(e)