Flask by example 10 (Testing the application)
Why testing is important
These are some of the reasons why testing in all it’s forms is important:
-
It helps developers make modifications and deploy code with more confidence. If all your tests pass for a new feature you just added to your app, you can be sure that the new feature did not break any existing functionality.
- Believe it or not, writing tests can help you structure your application properly which eventually leads to better code. Some developers even advise others developers to write tests before the main program, as it helps you envision how your functions/methods are going to look and work before you start writing the main program.
- It speeds up development time; at the beginning of a project the extra time taken to write tests slows the whole development down, but in the long run it eventually makes it faster as all the time that would have been spent debugging errors as a result of a change in codebase would have been discovered before that change was even made.
- There's this "Beast mode" feeling i get anytime my tests pass (especially on Open source projects i contribute to)...Lol
To start our tests, we’re going to install a test runner for python called nose:
you can use the de-facto unitTest
module if you want to, but nose comes with a lot of sweet features. I really love the autodiscover
feature which automatically discovers test files in your application and runs them. the filename just has to conform to this regex ((?:^|[\\b_\\.-])[Tt]est
.
Create a new file named api_test
with the following content and place it in the api
folder (You can place it anywhere you like, it’s up to you).
Before the tests are run, we need to setup some basic variables or configuration for our application, this is done by defining the setUpClass
and/or setUp
method, the major difference between these two methods is that setUpClass
is called only once throughout the duration of the tests while setUp
is called before each test is run.
So use setUpClass
to set up properties or variables that you want to be accessible to all the test methods in your class which may be too expensive or time-consuming to re-create anytime a new test is about to be run.
Also setUpClass
is a class method, so it must be decorated with @classmethod
In the setUpClass method:
I set DEBUG
to False to prevent Flask from starting two processes when we’re trying to test the application (This is how it’s able to detect changes and automatically reload itself when you’re running it in debug mode)
While TESTING
was set to True for better error propagation.
celery.conf.update(CELERY_ALWAYS_EAGER=True)
with this we don’t need to start celery from the command line, the task would be run as if it was a normal function.
We also created a session with requests.Session
, so we can login to the application and perform some our tests as a logged in user.
At the end of everything, we started a new instance of our application which we would run our tests against.
The small pause (sleep(2)
) at the end of the method is important, so the flask application has enough time to initialize and listen for connections before we start running our tests, if you don’t do this most of the tests won’t pass as requests
will be unable to connect to the application.
Without any explanation, you can guess what the tearDownClass
method does. There is also another method that wasn’t used here tearDown
which is run at the end of every individual test.
Now that we’ve got that out of the way, you can run the tests with:
nose should be able to discover the file and run it. You should see something like this in the terminal
If you’re paying attention you might have noticed the weird name of the last test test_zelery
, this was done intentionally to make the celery test run last. Why?
Because nose and even unitTest run the test alphabetically, which means if we named the test test_celery
it would have been run first, and at that point we wouldn’t have had any poll in our database which means the test would fail.
Technically what we’ve done isn’t unit testing because one of some of our tests depend on each other before they can be run. Tests are supposed to be independent of each other, the order of execution shouldn’t matter.
What advantage is there to this?
- This means that your unit tests can be parallelized (especially if you have a lot of unit tests)
But does that make our tests useless?
- No sometimes you actually want to test like this, and follow a particular process to test something bigger, so it’s perfectly fine if that’s what you intended.
How can we make this a true unitTest?
There are two things, we can do:
The first one is to make it a monolithic test and include the code that creates the poll inside the test_celery
method, but this has another downside, if the code that creates the poll fails or raises an exception for some reason, it would be hard to track down the exact error that made the tests fail.
The second option and the cleanest approach IMO is to move the code that creates the poll into setUpClass
and have the poll created at the before any test is even run.
This is the second approach:
Perfect unit tests and no funny names :)
Test Coverage
Now another question arises, how do you know that your application is well tested and that your unit tests reach all the important parts of your code?
There is a nice python module that does that and more Coverage.py
It can be installed from pip with:
nose makes it easy to use coverage, just run the tests with the --with-coverage
flag
At the end of the tests you should see a nice output that shows you the number of statements, the number of lines missed and the percentage covered, at the bottom there is also a overall summary of these information.
When i ran this, i got a coverage of 37%, you should get something similar or better if you wrote some extra tests or even worse if you added a lot of untested code.
Coverage.py even provides an option for displaying the test results as html:
a new folder called htmlcov
should be created in the working directory, inside the folder you’ll find html files of all the python files in your project, our api.py
file should be named api_api_py.html
Open it up, you should see something like this:
hmmm 18% coverage pretty poor right? Yeah, this poor result boils down to the fact that we didn’t actually test the methods (call them in our tests), we only tested the routes which in turn called the required methods. You’ll even notice that the Lines containing @api.route
are all highlighted with green.
There is no recommended percentage for coverage, so don’t get carried away by the numbers, a high coverage percentage means the code is well tested, but most times people get carried away and turn their code into a horrible dissected mess just because they want to get better coverage results.
Imagine us doing this, so we can increase our test coverage by exposing a new function called poll_vote
:
Don’t do this, First of all the method isn’t re-usable, so it shouldn’t even be exposed to the outside world, it’s more appropriate to make it an inner method of api_poll_vote
(if you really want to make it new function). If you continue like this, you’ll end up having a lot of localized functions splattered everywhere that eventually make the code harder to read that it was before.
Don’t sacrifice readability for better test coverage
Not every part of your code needs to be tested, some parts are simply irrelevant or are handled by an underlying library. It would make no sense to write tests for SQLAlchemy in your code.
If you noticed, coverage.py ran tests for a whole lot of files we didn’t care about like config.py
and some libraries that we made use of e.g SQLAlchemy
and even requests
!. These extra reports reduced the overall coverage percentage of our code, so you’ll probably want to exclude them.
Excluding those unimportant files, the actual code coverage at the time of writing is 57.50% and that’s quite decent IMO.
You can exclude files and directories with coverage.py
but that’s out of the scope of this tutorial, The Coverage docs has more information about that.
Aside unit tests, they’re still a lot of tests to be carried out on the application:
-
- Load testing
-
- Functional testing
-
- Acceptance testing etc
but unit tests are at the top of the pile, they’re one of the most important tests carried out in the Life cycle of a software project and are mostly written by you the developer who understands the inner-workings of the code.
Non developers or those that aren’t familiar with the codebase can still write tests for your code with a tool library Cucumber
It should be noted that Flask has an inbuilt testing client, i decided to use requests because it’s easier to get json responses from the api and i also wanted to test against a real instance of the application.
In the next part, we’re going to round of the series by deploying our application to Heroku and instead of SQLite, we’re going to use PostgreSQL.
Thanks for reading.
Don’t forget to share this part if you enjoyed it. Cheers!