Omar Abou Mrad

Upgrading your projects' dependencies requires too much effort, it has to be handled carefully and more often than not, tends to drastically fail. We've not-so recently decided to migrate one of our projects out of Django 1.6.X onto 1.8.5 and despite the amazing pluses we get from django 1.8, we were hit with big slap in the face with django migrations.

Despite the extremely slow process of creating the database and applying all the migrations, shortly after firing the migrate command during the Rendering model states we were greeted with:

Segmentation fault (core dumped)

It was simply impossible to debug this issue within our project because we deal with not dozens, but hundreds of models (tables) as well as heavy relationships between these models. What we needed to do is try and duplicate this issue outside the scope of the project and in an isolated environment with bare minimum dependencies. Furthermore, the models should be random in order to properly test against different schema sizes.

Given the above requirements, I built a django random app generator that pours out a massive number of apps, models and fields in order to properly test the migrations on a project marginally larger from a database schema perspective than others.

The test-app-builder can be found here alongside instructions on how to run it. The target project the upcoming tests were running against can be found here.

The Setup

Here's the setup for the tests being run.

% vagrant --version
Vagrant 1.7.4
% VBoxManage --version
% grep 'box =' Vagrantfile = "ubuntu/trusty64"

The Vagrantfile is the absolute default built by the vagrant tool.

Test #1 - Against Django 1.8.6

It turned out that the issue was pre computing the model states consumed way too much memory. The segmentation fault error could be solved in two different ways:

Test #2 - Against Django 1.9 Beta1

Thankfully, Django 1.9 Beta1 has resolved this issue and life is good again! The relevant section from the 1.9 release notes states the following concerning this matter:

"When applying migrations, the 'Rendering model states' step that's displayed when running migrate with verbosity 2 or higher now computes only the states for the migrations that have already been applied. The model states for migrations being applied are generated on demand, drastically reducing the amount of required memory."

This improvement also comes with its own requirement, though this is a major relief!

Thank you Django for fixing this. <3


comments powered by Disqus

Copyright 2015 - licensed under CC BY-NC-SA