How to create a progress bar when initializing a DataFrame?

Question

I want to get the number of rows every time a new one is created, when I upload a file .csvto the data framework:

def file_len(fname):
    with open(fname) as f:
        for i, l in enumerate(f):
            pass
    return i + 1

csv_path = "C:/...."
max_length = file_len(csv_path)

data = read_csv(csv_path, sep=';', encoding='utf-8')

With this code, I get the maximum number of rows, but I don’t know how to get the number of rows in the data frame, every time it is created. I wanted to use them to make a progress bar at 0-100%

Answer 1

You cannot do this - you will need to change the function read_csvand possibly other functions in pandas.

---

EDIT:

It seems now this can be done with chunksize=rows_number.

iterator=True - , , .

Jeff

import pandas as pd

from StringIO import StringIO

data = """A,B,C
foo,1,2,3
bar,4,5,6
baz,7,8,9
"""

reader = pd.read_csv(StringIO(data), chunksize=1)

for x in reader:
    print x
    print '--- next data ---'

:

     A  B  C
foo  1  2  3
--- next data ---
     A  B  C
bar  4  5  6
--- next data ---
     A  B  C
baz  7  8  9
--- next data ---