Moving a large dataset through django databases

Question

What is the best way to move a large dataset from one django database to another?

I would like to do this at a high level (in django), not at the database level. All existing tools that I know (dumpdata / loaddata, classizer, django-extensions) are stored in memory, so it does not process large data sets.

Answer 1

I found this post very useful - Migrating Django from MySQL to PostgreSQL is an easy way .

Answer 2

/ , script, django dumpdata, , MemoryError

Script https://github.com/fastinetserver/django-dumpdata-chunks

:

1) :

mkdir some-folder

./manage.py dumpdata_chunks your-app-name
--output-folder=./some-folder --max-records-per-chunk=100000

2) :

find ./some-folder | egrep -o "([0-9]+_[0-9]+)" | xargs ./manage.py loaddata