Django migration killed

I am pretty sure about Django, but until recently, I relied heavily on generated migrations. I wrote a small user migration, and shortly after my CI started complaining about timeouts, and ultimately this is due to migrations from Django during deployment.

At first I was able to fix this problem, but I don't know what I did (if anything) that fixed it. The problem seems to be related to some special code that I entered for a specific migration. Here is what I know:

  • Initially, everything was fine, but the migration began to work very long (relatively) after adding my custom code. About 10 seconds.
  • It works sometimes. i.e. If I started the migration ten times from the command line, sometimes it will work, and sometimes it will fail.

The output is as follows (application names edited):

[ web@dev myapp]$ ./manage.py migrate Operations to perform: Apply all migrations: myapp1, myapp2, myapp3, myapp4 Running migrations: Killed 
  • At first I thought it was because I use RunPython to run a Python function that copies data between two fields before deleting one of the fields. The documentation hinders its use for PostgreSQL, but is there a better way to do this?
  • The business scenario here is that I had a logical field that I needed to switch to a set of parameters (CharField with options ). The code checks to see if the boolean value is true and sets the correct value for the character field. I have done it twice. The first time it was over, but I have not tested it in another database yet.

This is the migration (application names edited):

 from __future__ import unicode_literals from django.db import migrations def fix_consulting(apps, schema_editor): my_model = apps.get_model("myapp", "MyModel") for m in my_model._default_manager.all(): if m.consulting: m.detail = "CONSLT" m.save() class Migration(migrations.Migration): dependencies = [ ('myapp', '0024_auto_20160117_1113'), ] operations = [ migrations.RunPython(fix_consulting,atomic=False), ] 

My thoughts:

  • Perhaps the code I'm writing here has been working for too long? There are less than a hundred models in the database, so I don’t know why the fix_consulting function fix_consulting take so long.

  • If I add print instructions at the beginning of fix_consulting , they will be executed only occasionally and will be killed at another time. Be that as it may, I ran it 6-8 times, and he was killed every time, but at different points

Additional Information: - Using Django 1.9 - Using PostgreSQL 9.4.4 - The error occurs mainly on CentOS, but also OSX

+5
source share
1 answer

I believe that your problem was caused by the amount of data that you might need to cache when using all , since this returns all instances of the object, so you can filter at the database level before returning the objects, because you only need to change the values ​​of the field that you also can do at the database level. In general, this will change your code to the following.

 def fix_consulting(apps, schema_editor): my_model = apps.get_model("myapp", "MyModel") my_model._default_manager.filter(consulting=True).update(detail="CONSLT") 

This puts memory management responsibilities in a database that seems to have solved your problem.

Going forward, I would recommend always filtering out what is returned from db only to what is really needed (whether by splicing or filtering)

+7
source

Source: https://habr.com/ru/post/1240812/


All Articles