Background/requirements
This post assumes you have some familiarity with python mocking and the mock
library. If not, this intro by Naftuli Kay and this other intro by Amos Omondi are great places to start; these articles are where I figured out most of the mistakes I’ve been making. I’m working in Python 2, which still has mock
v 2.0.0 as its own library rather than part of unittest
, but the mistakes below also apply for Python 3.
Launch your business in minutes with GoDaddy Airo™
Example mistake 1
For the first example, let’s say we’re testing a function is_cat_person
, within the module main.py
. The function loads some json from a file using json.load
and decides whether or not the person represented by the json is a Cat Person:
# main.py
import json
def is_cat_person(filepath):
with open(filepath) as f:
person_json = json.load(f)
...
# Do some stuff with person_json and return True if Cat Person, False otherwise
We want to test the function, but we don’t want to create and delete a temporary file representing our test Cat Person, so we’ll mock json
.
# test_example_1.py
import unittest
from mock import patch
from main import is_cat_person
class TestIsCatPerson(unittest.TestCase):
@patch('json')
def test_returns_true_for_cat_person(self, mock_json):
mock_json.load.return_value = {
'meyers_briggs_type': 'INTJ',
'likes_laser_pointers': True,
'dresses_like_a_cat': True
}
self.assertTrue(is_cat_person('path/to/person'))
Why doesn’t this work?
Show error message
Error
Traceback (most recent call last):
...
File "./venv/lib/python2.7/site-packages/mock/mock.py", line 1522, in _get_target
(target,))
TypeError: Need a valid target to patch. You supplied: 'json'
Show the mistake
As the traceback indicates, mock.patch
can't find the json
module. This can be fixed by changing the patch line to @patch('main.json')
This fits with the rule of thumb from the documentation that you should "patch where an object is looked up, which is not necessarily the same place as where it is defined." However, in this case, that isn't the full story: the specific error produced within the mock library is caused by any patch that doesn't involve a .
as a separator (the same is true in Python 3's unittest.mock library). If we were to instead use @patch('json.load')
, modifying the return value statement to remove the redundant .load
, we'd be breaking the rule and mocking it where it came from, but it still works: we're mocking the function from the json
package, but main.py
's import points to the whole package. However, if main.py
is rewritten to use from json import load
, and we tried to patch json.load
instead of main.load
, our test would fail, since we wouldn't be patching the function we're using.
Example mistake 2
Now we’ve modified our is_cat_person
function slightly to include a call to a validate
function, which checks that the json we’ve loaded has a valid validation_id
property, throwing an exception if it doesn’t:
# main.py
import json
def is_cat_person(filepath):
with open(filepath) as f:
person_json = json.load(f)
if validate(person_json.get('validation_id')):
...
# Do some stuff with person_json and return True if Cat Person, False otherwise
Let’s say the validation function is time-consuming, so we modify the previous test to mock validation:
# test_example_2.py
import unittest
from mock import patch
from main import is_cat_person
class TestIsCatPerson(unittest.TestCase):
@patch('main.json')
@patch('main.validate')
def test_returns_true_for_cat_person(self, mock_json, mock_validate):
mock_json.load.return_value = {
'meyers_briggs_type': 'INTJ',
'likes_laser_pointers': True,
'dresses_like_a_cat': True,
'validation_id': 'h19d8w22'
}
self.assertTrue(is_cat_person('path/to/person'))
What’s wrong with this?
Show error message.
In this case, the test fails, but there is not necessarily any error beyond AssertionError: False is not true
.
Show the mistake.
The problem is the decorator order: as things stand, mock_json
is mocking main.validate
, and mock_validate
is mocking main.json
. Decorators move outward/upward from the decorated function, so the arguments need to have that order. When I set this up, there was no error, because the person object returned by mock_validate
was taken as truthy, and the True
that was returned by mock_json
was found to not be a cat person.
Side note: I'm currently reading and enjoying Clean Code as part of our engineering book club, and I've been thinking about the way that some of these error-prone mocking situations come about in part because the functions we're testing are too long and/or work at multiple levels of abstraction. Why should one function be responsible for loading json from a file, validating that json, and also performing a classification? But we'll stick with it for now, since it's convenient for these examples.
Example mistake 3
Suppose we now want to do a more comprehensive test, in which we check the logic of how the function deals with valid and invalid input, using our own simplified version of validation.
# test_example_3.py
import unittest
from mock import patch
from main import is_cat_person
def simplified_validate(validation_string):
if validation_string == 'h19d8w22':
return True
else:
raise Exception('Invalid validation id')
class TestIsCatPerson(unittest.TestCase):
@patch('main.validate')
@patch('main.json')
def test_validation_logic_flow(self, mock_json, mock_validate):
mock_validate = simplified_validate
mock_json.load.return_value = {
'meyers_briggs_type': 'INTJ',
'likes_laser_pointers': True,
'dresses_like_a_cat': True,
'validation_id': 'h19d8w22'
}
self.assertTrue(is_cat_person('path/to/person'))
mock_json.load.return_value = {
'meyers_briggs_type': 'INTJ',
'likes_laser_pointers': True,
'dresses_like_a_cat': True,
'validation_id': 'zzzzzzzz' # Should trigger a validation exception
}
self.assertRaises(Exception, is_cat_person, 'path/to/person')
Show error message
The first assertion passes, but the second doesn't. There's no meaningful error, other than the fact that an exception isn't raised. A good IDE might call your attention to the mock_validate
argument in the two places where it's used.
Show mistake
It looks like this is setting up simplified_validate
to be a stand-in for mock_validate
, and by extension a stand-in for main.validate
, but the substitution isn't taking place. Instead, we want to use mock_validate.side_effect = simplified_validate
, or to simplify things visually, we could change the patch line to @patch('main.validate', side_effect=simplified_validate)
and omit the mock_validate
argument. I find the side_effect
terminology a bit confusing, since the term doesn't do anything to capture the fact that this side_effect
will in effect be subbed in for the mocked function and called with the same arguments. If anyone has a good way of explaining this, I'm all ears.
Example mistake 4
Let’s revisit example 2, but with two differences. First, we’ve switched to using a Validator
object, vs a simple validate
function:
# main.py
import json
def is_cat_person(filepath):
with open(filepath) as f:
person_json = json.load(f)
if Validator().validate(person_json.get('validation_id')):
...
# Do some stuff with person_json and return True if Cat Person, False otherwise
Second, this time we’ll dodge the patch-ordering issue completely by specifying that our mock Validator will be a general MagicMock
instance:
# test_example_4.py
import unittest
from mock import MagicMock, patch
from main import is_cat_person
class TestIsCatPerson(unittest.TestCase):
@patch('main.json')
@patch('main.Validator', MagicMock())
def test_returns_true_for_cat_person(self, mock_json):
mock_json.load.return_value = {
'meyers_briggs_type': 'INTJ',
'likes_laser_pointers': True,
'dresses_like_a_cat': True,
'validation_id': 'h19d8w22'
}
self.assertTrue(is_cat_person('path/to/person'))
Show error message
No error message, and the test passes.
Show mistake
This is a subtle/debatable one. Whether or not this is ok hinges on how much we depend on this test to tell us if something changes about how the Validator
object is defined and used in main.py
. By making it a MagicMock
instance, it will continue working even if the Validator
method called within is_cat_person
changes or stops existing. This is one drawback of the flexibility of MagicMock
. The alternative is to import the Validator object and use the mock library's create_autospec
function, as in: @patch('main.Validator', create_autospec(Validator))
.
Those are all the big ones I’ve run up against so far. Happy mocking!
(Kitten image source, CC0 license)