Python csv reader does not read all lines

So, I have about 5008 lines in a CSV file, total 5009 with headers. I create and write this file in the same script. But when I read it at the end, either with pandas pd.read_csv, or with the python3 csv module, and printed len, it prints 4967. I checked the file for any weird characters that python can confuse but not see Any. All data is separated by commas.

I also opened it in an elevated form, and it shows 5009 lines not 4967.

I could try other methods from pandas, such as merge or concat, but if python doesn't read csv correctly, it is useless.

This is one of the methods I've tried.

df1=pd.read_csv('out.csv',quoting=csv.QUOTE_NONE, error_bad_lines=False)
df2=pd.read_excel(xlsfile)

print (len(df1))#4967
print (len(df2))#5008

df2['Location']=df1['Location']
df2['Sublocation']=df1['Sublocation']
df2['Zone']=df1['Zone']
df2['Subnet Type']=df1['Subnet Type']
df2['Description']=df1['Description']

newfile = input("Enter a name for the combined csv file: ")
print('Saving to new csv file...')
df2.to_csv(newfile, index=False)
print('Done.')

target.close()

Another way I've tried is

dfcsv = pd.read_csv('out.csv')

wb = xlrd.open_workbook(xlsfile)
ws = wb.sheet_by_index(0)
xlsdata = []
for rx in range(ws.nrows):
    xlsdata.append(ws.row_values(rx))

print (len(dfcsv))#4967
print (len(xlsdata))#5009

df1 = pd.DataFrame(data=dfcsv)
df2 = pd.DataFrame(data=xlsdata)

df3 = pd.concat([df2,df1], axis=1)

newfile = input("Enter a name for the combined csv file: ")
print('Saving to new csv file...')
df3.to_csv(newfile, index=False)    
print('Done.')

target.close()

, CSV - , , .

: , ...

Edit2: nrows , 4000 . , 5000 , 4967.

Edit3: csv , , , 5008 . python csv ?

+4
2

. , , - .

, , :

GO:0000026  molecular_function  "alpha-1
GO:0000027  biological_process  ribosomal large subunit assembly
GO:0000033  molecular_function  "alpha-1

. ( , , csvreader, . , - !)

, .

: , :

quotechar=None
+1

, , , , , - , foo,bar.

error_bad_lines=True. Pandas : http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html, , , , 41,

error_bad_lines: boolean, True (, csv ) , DataFrame . False, " " DataFrame. ( C-)

csv.QUOTE_NONE, -, escape_char + , , , , . https://docs.python.org/3/library/csv.html#csv.Dialect

0

Source: https://habr.com/ru/post/1650686/


All Articles