Five Advanced Pytest Fixture Patterns
The pytest package is a great test runner, and it comes with a battery of features — among them the fixtures feature. A pytest fixture lets you generate and initialize test data, faked objects, application state, configuration, and much more, with a single decorator and a little ingenuity.
But there’s more to pytest fixtures than meets the eye. Here’s five advanced fixture tips to improve your tests.
Factory Fixture: Fixtures with Arguments
Passing arguments to a fixture is the first thing people want to do with fixtures when they start using them.
But initializing hard-coded data is the precursor to that, and it’s what the
@fixture decorator excels at:
But because pytest is just passing objects around – with a sprinkling of magic to make it all work properly behind-the-scenes – you can return just about anything — including a factory that lets you control how your test data is initialized by passing arguments with the values you want:
This is a very powerful pattern; the new fixture, now named
make_customer to make it obvious to someone that it’s a factory that makes something, lets you override the
last_name parameters instead of hardcoding them. But if you don’t care what they are in a particular test, you can leave them out.
How it works is like this: instead of returning an instance of
Customer (like the first example demonstrates), instead it returns a function (called
make) that does all the heavy lifting for us. That function is the factory; it’s responsible for creating the ultimate object and return it.
- Clearly name your factory fixtures
You can absolutely use generic names, like
customer, for factory fixtures, but I recommend you do not. Instead make it clear the user of the fixture – it may well be someone other than you – should instantiate it first and not just assume it returns an instance of
- Initializing a fixture with static values is easy, but passing arguments requires a function
You can create fixtures with arguments by returning a function — that function is called a factory.
Consider two separate fixtures in a test suite for a fictional ecommerce store. Now let’s say you want to represent the concept of a transaction – i.e., a customer having completed a sale – you could do this by creating fixture like this:
But instead of repeating yourself unnecessarily, you can instead take advantage of the fact that pytest fixtures can, in turn, depend on other fixtures:
This fixture takes a mandatory
transaction_id and an optional
sale. If the latter two aren’t specified, they’re created by the fixture automatically by calling their respective fixtures.
Two-way data binding with closures and
Occasionally, you need to mock, fake or stub out parts of your code base to ease the testing of other parts of your code. Usually to achieve a certain goal, like simulating rare error conditions, or scenarios that you cannot easily reproduce without such techniques.
monkeypatch feature in pytest lets you do this with a test, but you can also do it in a fixture. When you patch out a piece of code you may want to check that it was at least called with the expected parameters. That is especially important if the patched function is nestled deep inside other pieces of logic that makes it hard or impossible to query directly.
That alone is why a lot of developers opt to put the monkeypatched code in a test instead of a fixture: you can directly control the patched code and interrogate its state, but there’s an easy way to do the same with a fixture:
Here I’m patching out a real
send_email function somewhere in our fake ecommerce codebase. Let’s pretend it returns
True if it sends an email. But if you want to check that it was called, you need to do a bit more leg work. So, I patch it out with a mocked version that captures the sent emails into a list
sent_emails. But I also return that same list!
This approach works because:
_send_emailmock function is lexically binds
That means the
_send_emailfunction holds on to the
sent_emailsobject even though scope of the function changes when it’s patched into our “real” ecommerce codebase at
- Everything is an object
Which, of course, includes both functions and lists. So when I return
sent_emailsI am in fact returning the selfsame list object that our test can then access.
The end result is that I can query the
sent_emails list whenever I need to.
Let’s say I want to test that
commit_order can take an instance of
Transaction – which contains a reference to a
customer and a
sale along with a
transaction_id – and send an email (and whatever housekeeping you’d expect it to do in a real ecommerce system):
As you can see, the list object is available and its state changes whenever it’s mutated by
_send_email, the mock function in the fixture. Now in this example the flow of data is one way: from mock function to the test caller, but you can easily reverse it, and modify the list in the test and have its changes reflected back into the mock function.
monkeypatchis itself a fixture
You can abstract frequently monkeypatched parts of your code into a fixture and use that instead. It’s a great abstraction tool and centralizes something that is very easy to mess up in ways that can be fantastically hard to debug.
- You can combine this pattern with the factory pattern
And gain the benefits of simplifying complex or tedious instantiation patterns plus the two-way data binding.
- Two-way data binding with closures is a useful tool in your toolbox
The example is trivial, of course, but for complex hierarchies of objects or state, the ability to track the changes made throughout the system is a useful pattern. And naturally this idea is generalizable to not just fixtures or test code.
- The existing
unittest.mockcan do this also
If you prefer the classic
unittest.mockapproach you can do it with
MagicMock.assert_called_once_with()and friends. However, I prefer this approach, because it’s explicit. The only magic is the patching; the rest is just Python. And there are no limits to what you can write in your patched function, nor what you return or how you choose to formalize the contract between fixture and test.
Tear down and Setup Fixtures with
yield inside a fixture you can return a scoped object or value at that point in time, and pytest will only resume the generator once the test is complete. You can use this pattern to create traditional setup and teardown patterns:
Here I create a connection object,
open() it, and then
yield it to the test. When the test exits – for any reason – the
finally clause is run and then I
close() the connection.
One unfortunate problem with this approach is that it does not work with the factory pattern. To work around that, you can combine them using pytest’s
Like the factory pattern from earlier, I return an instantiated connection with the
port parameter values drawn from
make. I open the connection – as before – but to ensure the cleanup happens when the test is finished, I add
cleanup to pytest’s
It’s a bit more complicated, but it ensures a clean separation between creation and destruction of a resource.
Now you might wonder why I couldn’t just
yield from inside
make? Well, you can, and it works OK… but you won’t get automatic cleanup when the test exits. The reason is that pytest only cleans up the
make_db_connection fixture if it’s a generator function, but it isn’t… so it doesn’t.
yieldto manage teardown and setup of application state, like database connections or files
It works well if you don’t need the factory pattern, and you can yield whatever you like: a tuple of objects, if that’s what you need.
I recommend you wrap the setup and teardown in
finallyeven though there are some guarantees made by pytest that it’ll try and clean things up if pytest crashes.
- You can use
request.addfinalizerif you have especially complex requirements
It works with everything, and you can call it from tests, too, if you have to. It’s also the simplest way to pair factory patterns with other patterns I have demonstrated, like two-way binding.
Triggering Side-Effects and Errors with
unittest.mock library, the
monkeypatch tool is surprisingly simple and lightweight. Part of the reason is that you can freely use parts of the existing
mock library as you like; but I am of the opinion that it’s simply a different approach to mocking and patching than what you get with
mock library has a large, and complex, set of features and quirks that most of us that work professionally with Python grew accustomed to before pytest was a thing. But if you can keep things simple and solve the problem cleanly with
monkeypatch, you should.
Let’s go back to the ecommerce order system. Let’s say we want to test – from the perspective of the ecommerce’s frontend, say the UI or REST API – what happens if
commit_order(transaction) triggers an error, somehow. Now let’s also pretend that this is the main entrypoint into the order system: it does all the database stuff; it sends emails; it checks and updates stock inventory — it’s got a lot of moving parts, and it’s an important cog in the machinery.
Let’s flesh out a couple of potential errors to simulate:
An item sold out in the time it took the user to click “order now” and before the system could reconcile its inventory. Or maybe there’s a human on the other end in the fulfillment center that updates a transaction with “out of stock”.
A duplicate transaction made its way through, somehow, due to an accidental double-buy. There’s a check in place to prevent duplicate entries in the database, like database-level constraints, for instance.
In that case we need a way to test that these scenarios are handled correctly. Ordinarily you’d write an exhaustive test suite to reproduce them, and that’s great and all, but maybe you’re testing other parts of the code, like we are here, and how they interact with error states. Or maybe you’re building a fixture that can trigger these cases arbitrarily, so other developers in your team can make use it for other test cases. Either way, it’s the same situation.
So what you’d want is a fixture specifically designed to patch key parts of
commit_order – not the whole thing – to induce these two error scenarios. What I’d want is a feature switch to test before/after conditions and ensure the system correctly handles them all.
The example above accomplishes this by reusing several patterns from earlier. Instead of a list, it’s a dictionary with the state the system is supposed mirror. Substitute this for any number of situations in your own codebases.
Now a matching test:
I think it speaks for itself. By modifying
state I can induce an error by flicking a switch. And, indeed, the mock function abides and raises
OutOfStockError as we’d expect.
By selectively patching only the parts of the code that needs patching you minimize the likelihood that your patches are overreaching. An all too common occurrence in real life. You could easily do this with multiple fixtures, each with and without an error state, and multiple tests. But if you have a sufficiently complex set of interactions that may not be feasible or maintainable.
You could, of course, do this with the
side_effect feature in
I like this approach because it hews closer to the idea that we swap out function-for-function and that the function has a minimal amount of logic that you can modify over time as your requirements grow. It’s all too easy to end up daisy-chaining
Mock() objects only to have to refactor all of it if the structure changes slightly. Or, worse, due to how “flexible”
Mock objects are, your code may have fundamentally changed and broken in some way, but your test stays green!
- Fixtures are not just there to initialize simple objects
But, of course, if you don’t need more than that — good. Simplicity is a good thing. But, if you work on large or complex codebases, that may not be enough.
- You, the developer, determine the relationship and contract you have with a fixture
I’ve shown that you can use two-way binding to effect changes inside a mocked function; but, it adds complexity. I find the complexity manageable when the alternative is a dozen tests, each just different enough to warrant a new test or new fixture to support it, but without the impetus for anyone to try and refactor how they test things to begin with.
Two-way binding and factory patterns go a long way towards solving some of the sprawling test suites you end up with in real codebases.
- Don’t forget
monkeypatchfixture is intentionally simple, possibly as a cautious overreaction to how confusing and complicated
mockcan be. But the silver lining is that it forces Python developers to trade magic for explicit code, even if lexical scoping and mutability adds its own complexities.