If you use Test Driven Development or test your code regularly during development, running your test suite can start to become very time consuming. Here’s some tips for speeding up your tests. All of these changes can go in a test_settings.py file, which your can import into your settings.py file only when a test is being run.
- Use Sqlite
I’ve written about this before. When using Sqlite, the Django test runner will create the DB in memory instead of on disk, which is faster than any other database available.
- Skip South Migrations
If you’re using South, migrations will be run during the set up for every test, which can add a lot of time to your tests. Set “SOUTH_TESTS_MIGRATE = False” to use the normal syndb behaviour for tests.
- Use an In-Memory Storage Class
There’s a project called django-inmemorystorage, which stores files in memory instead of on disk. This is not only faster but eliminates the problem of having to clean up files created during testing. Unfortunately, it doesn’t currently support the url() method, so using the “.url” attribute of a FileField or ImageField will throw an error, which is why I created a fork to solve this issue. After installing set “DEFAULT_FILE_STORAGE = ‘inmemorystorage.InMemoryStorage'” in your test settings.
- Use Weaker Password Hashing
Prior to Django 1.4, Django used SHA1 to hash passwords. Starting with 1.4, Django can be configured to use a number of password hashers. The default setting is:
PASSWORD_HASHERS = (
Django will encrypt passwords using the first one in this list, PBKDF2PasswordHasher, which was designed to take a really long time to execute in order to be harder to break. This is an important security feature that should be left in place on your live site, but for running a test suite it’s really not needed. You can set it to the following in order to go back to using SHA1:
PASSWORD_HASHERS = (
I have one project whose tests took 50 seconds to complete before I made this change, and 7 seconds afterwards.
- Mock Network Calls
Mocking network calls is not only a best practice, but is also a lot faster than making real requests. I recommend using Mock. It gets the job done, it’s widely used, and it’s even being merged into the standard library of Python 3.3. Here’s a condensed example of how to use it:
Please note, aside from using mock objects, these techniques should only be used while testing during development. On a regular basis (such as nightly, or after every commit on a CI server, and/or before pushing your changes to your code repo), you should run your test suite using settings as close to your live site settings as possible (same DB type, same storage, run migrations, etc.) in order to weed out any bugs.
If you have any tips feel free to share!
Another problem I’ve encountered with TastyPie is that a 500 Internal Server error is thrown when the user sends bad input. One of my clients was using hurl.it to try out an API that I wrote, using hand written JSON. Ocassionally there would be mistakes, such as the use of single quotes instead of double quotes, or the inclusion of a trailing comma, which would cause JSON deserialization to fail. Instead of catching this error and returning a 400 Bad Request response, TatsyPie just lets the error propagate, resulting in a 500 Internal Server Error. Here’s some code I wrote to fix that:
In TastyPie, a library for creating REST APIs for Django projects, the recommended way to limit the objects available to a user is through the apply_authorization_limits() function. This method is extremely handy and easy to use, but has a serious flaw; by filtering out objects from the queryset that a user should not access, TastyPie acts as if that object doesn’t exist at all. This causes 2 problems:
- When trying to access an object which a user doesn’t have access to, a 404 Not Found status code is returned instead of the more semantic (and less misleading) 401 Unauthorized. Normally you wouldn’t care about misleading users trying to access other users’ data, but this caused a bit of confusion with one of my clients who has another team writing a mobile app using the API I wrote. When trying to access an object while logged in as the wrong user the API tells them the object doesn’t exist, instead of telling them they don’t have access.
- Sometimes your get a 500 Internal Server error. If a user tries to update (PUT) an existing resource for another user, instead of returning a 401 status code, TastyPie thinks the object doesn’t exist, tries to insert a new row in the database with an already used primary key, and an IntegrityError is thrown. Even though this only happens during incorrect usage, it still looks bad.
To solve this problem I created a CustomModelResource class with an overridden obj_get() method.
On web pages that contain embedded Flash content where you’re also using the jQuery Fancybox plugin, you can change the CSS visibility property to ‘hidden’ on the Flash content. Here’s a quick snippet I used to hide iframes from Youtube:
I just found out about this cool web app called Codecademy that teaches programming to complete novices. As soon as you visit the site you can start learning to program without having to create an account, and as of now it looks like it’s completely free. If you or anyone you know is interested in learning to program I recommend trying it out.
Afterwards, I went searching for the cause and found this forum post, which explains that the problem is caused by the Mozilla Archive Format extension, which has a setting that even affects pages not saved as .maff. To fix the problem, go into the MAF extension settings and change “When saving complete web page contents:” from “Take a faithful snapshot of the page” to one of the other options.