Why is my django database database so slow and often not working?

Question

I decided to use the django system model instead of coding raw SQL to interact with my database, but I have a problem that can certainly be avoided.

My models.py contains:

class Student(models.Model): student_id = models.IntegerField(unique = True) form = models.CharField(max_length = 10) preferred = models.CharField(max_length = 70) surname = models.CharField(max_length = 70) 

and I populate it, iterating over the list as follows:

 from models import Student for id, frm, pref, sname in large_list_of_data: s = Student(student_id = id, form = frm, preferred = pref, surname = sname) s.save() 

I really don't want to keep this in the database every time, but I don’t know of any other way to make django remember it (I would prefer to add all the lines and then make one commit).

There are two problems in the code.

• It's slow - about 20 students are updated every second.

• It does not even do this through large_list_of_data, but instead throws a DatabaseError message, "unable to open the database file." (Perhaps because I'm using sqlite3.)

My question is: how can I stop these two things? I assume that the root of both problems is that I have s.save (), but I don’t see a way to easily bring students to them, and then save them in one commit in the database.

Answer 1

Therefore, it seems to me that I should have looked harder before asking a question.

Some of the solutions are described by fooobar.com/questions/73594 / ... (the winning answer is to use django.db.transaction.commit_manual), as well as aggregated ones.

Other ideas for speeding up this type of operation are listed at fooobar.com/questions/73594 / ....