Why is my loop rewriting instead of adding?

I have several (25k) .csv files that I am trying to add to an HDFStore file. They all have the same headings. I use the code below, but for some reason, whenever I run it, the data framework is not added with all the files, but rather is only the last file in the list.

filenames = []  #list of .csv file paths that I've alredy populated
dtypes= {dict of datatypes}
store = pd.HDFStore('store.h5')
store.put('df', pd.read_csv(filenames[0],dtype=dtypes,parse_dates=
["date"])) #store one data frame

for f in filenames:
    try:
        temp_csv = pd.DataFrame()
        temp_csv = pd.read_csv(f,dtype=dtypes,parse_dates=["trade_date"]) 
        store.append('df', temp_csv)
    except:
    pass

I tried to use a subset of the list of file names, but always get the last entry. For some reason, the loop does not add my file, but overwrites it every time. Any advice would be appreciated, as it made me work. (python 3, windows)

+4
source share
2

, :

store.append('df', temp_csv)

, , "df" , .

0

/ DataFrame , , @SeaMonkey. , - .

filenames = []  #list of .csv file paths that I've alredy populated
dtypes= {dict of datatypes}

df = pd.DataFrame()
for f in filenames:
    df_tmp = pd.read_csv(f,dtype=dtypes,parse_dates=["trade_date"]) 
    df = df.append(df_tmp)

store = pd.HDFStore('store.h5')
store.put('df', df)
0

Source: https://habr.com/ru/post/1678900/


All Articles