Can someone explain the root of this index due to a range error?

Question

The following code causes a mysterious error with which I cannot find a solution. It works great when I tested it in a larger module, so I can’t understand why this is not working:

code

import csv with open('studentinfo.txt','a') as fo: #open the file in append mode (add to file, we don't wish to overwrite!) studentfileWriter=csv.writer(fo) #fo = file out (this can be called anything you like) id=input("Enter Student Id:") firstname=input("Enter firstname:") surname=input("Enter Surname:") test1=input("Enter test1 score:") test2=input("Enter test2 score:") test3=input("Enter test3 score:") studentfileWriter.writerow([id,firstname,surname,"Test1:"+test1,"Test2:"+test2,"Test3:"+test3]) print("Record has been written to file") with open("studentinfo.txt", "r") as f: reader = csv.reader(f) sorted_list = list(reader) # turn the reader iterator into a list sorted_list.sort(key=lambda x: x[2]) # use the third column as a sorting key print("\n".join(str(row) for row in sorted_list)) # prettier print 

Error message

 sorted_list.sort(key=lambda x: x[2]) # use the third column as a sorting key IndexError: list index out of range 

It is worth noting that the code works fine when there are no additions to the contents of the file. When adding a student to the file, the CORT does not work.

The original contents of the file

 001,Joe,Bloggs,Test1:99,Test2:100,Test3:1 002,Ash,Smith,Test1:20,Test2:20,Test3:100 003,Jonathan,Peter,Test1:99,Test2:33,Test3:44 

The contents of the file when adding the student being tested:

 001,Joe,Bloggs,Test1:99,Test2:100,Test3:1 002,Ash,Smith,Test1:20,Test2:20,Test3:100 003,Jonathan,Peter,Test1:99,Test2:33,Test3:44 006,Mulch,Cart,Test1:99,Test2:22,Test3:11 

The resulting error occurs at this stage (when a new student is added). The sort function otherwise works fine.

Update and clarification:

For training purposes, I need it to work as repl.it AND IDLE>

If someone can send repl.it as an answer (with my code above, working), which also works when it is implemented in IDLE with a txt file, I will accept as an answer.

Answer 1

The reason for your problem here is that you are not adding to csv correctly.

On Windows, the csv module has an error / limitation when working with Windows. It adds extra blank lines to each line (in fact, it adds an extra char carriage return) if you don't open the file properly. Therefore, to fix this:

Python 3:

 with open('studentinfo.txt','a',newline='') as fo: 

Python 2:

 with open('studentinfo.txt','ab') as fo: 

So, the csv module adds an extra \r to the end of your file. And, reading it again, it produces an empty string.

This works fine in repl.it because they use the python engine, which runs on the Linux sandbox), but the documentation still advises opening files, as I showed.

(the documentation on the csv module clearly speaks about this, even if he advises doing the same for read mode, and I never had a problem with a simple open("file.csv") )

See also my old question: portable way to write a CSV file in Python 2 or Python 3

If the double char carriage returns at the end of the file, you don’t see it (use Notepad ++ with "show all characters" to see the double character of CRCR), but csv.reader returns an empty string that does not work when sort uses your key function for comparison.

Now, if you want to be sure of it (because other people can edit your database, for example, using excel csv mode or other terrible things):

I would filter and sort and convert to a list at the same time using filter(None,...) , which removes the “false” (i.e. empty) lines:

 sorted_list = sorted(filter(None,reader),key=lambda x: x[2]) 

Of course, if the string contains only 1 or 2 elements, this will not work either. In this case, drop the filter , because we need to write lambda , and this is not in favor of understanding the generator:

 sorted_list = sorted((x for x in reader if len(x)>2),key=lambda x: x[2])