Separating type-specific code with singledispatch
Have you ever found yourself writing a litany of
if-elif-else statements peppered with
isinstance() calls? Notwithstanding error handling, they are often found where your code intersects with APIs; third-party libraries; and services. As it turns out, coalescing complex types – such as
pathlib.Path to string, or
decimal.Decimal to float or string – is a common occurrence.
But writing a wall of
if-statements makes code reuse harder, and it can complicate testing:
In this example I have a
convert function that converts complex objects into their primitive types, and if it cannot resolve the given object type, it raises a
TypeError. There’s also a keyword argument,
bankers_rounding intended for the decimal converter.
Let’s quickly test the converter to make sure it works:
Yep. It does. Remove the
default= argument and the
dumps function throws an exception because it does not understand how to serialize
But now I’ve trapped a number of independent pieces of logic in one function: I can convert data, yes, but how can I easily test that each conversion function actually does what it’s supposed to? Ideally, there would be a clear separation of concerns. And the keyword argument
bankers_rounding only applies to the Decimal routine and yet it’s passed to our shared
convert function. In a real-world application there might be dozens of converters and keyword arguments.
But I think we can do better. One easy win is to separate the converter logic into distinct functions, one for each type. That has the advantage of letting me test – and use independently – each converter in isolation. That way I specify the keyword arguments I need for the converter functions that need them. The
bankers_rounding keyword is not tangled up with converters where it does not apply.
The code for that will look something like this:
At this point I have built a dispatcher that delegates the act of converting the data to distinct functions. Now I can test the dispatcher and the converters separately. At this point I could call it quits, but I can get rid of the
convert dispatcher almost entirely, by offloading the logic of checking the types to a little-known function hidden away in the
functools module called
How to use
First, you need to import it.
In Python 3.7
singledispatch gained the ability to dispatch based on type hints which is what this article uses.
Much like the dispatcher from before, the approach used by
singledispatch is much the same.
The singledispatch decorator works in a similar way to the home-grown approach above. You need a base function to act as a fallback for any unknown types it is given. If you compare the code with the earlier example, this is akin to the
else portion of the code.
At this point the dispatcher cannot handle anything and will always throw a
TypeError. Let’s add back the decimal converter:
Note the decorator. The
singledispatch decorator transmutes the base function into a registry for future types you want to register against that base function. As I am using Python 3.7+ I have opted for type annotations, but if you do not wish to do this, you must replace the decorator with
The name of the function is
convert_decimal and sure enough it works on its own:
Now I can write tests for each converter and leave the messy type checking to
Simultaneously, I can invoke
convert with the self-same arguments and it works as you would expect: the arguments I give it are dispatched to the
convert_decimal dispatcher function I registered earlier:
Dynamically querying and adding new dispatchers
One useful side-effect of
singledispatch is the ability to dynamically register new dispatchers and even interrogate the existing registry of converters.
If you wanted to add the
convert_path function dynamically you can:
If you want a mapping of types to the underlying functions, the
convert.registry will show you what it supports:
You can also ask the dispatcher to tell you the best candidate function to dispatch to, given a type:
singledispatch function is useful, but it is not without limitations. Its main limitation is also apparent in its name: it can only dispatch based on a single function parameter, and then only the first. If you require multiple dispatching you will need a third-party library as Python does not come with that built in.
Another limitation is that
singledispatch works only with functions. If you need it to work on methods in classes you must use
singledispatchencourages separation of concerns
By separating the type checking from the converter code you can test each function in isolation and your code, as a result, is easier to maintain and reason about.
- Converter-specific parameters are separate from the dispatcher
This ensures that, say,
bankers_roundingis declared only on converters that understand it. That makes the function signature easier to parse for other developers; it greatly improves the self-documenting nature of your code; and it cuts down on bugs as you cannot pass invalid keyword arguments to functions that do not accept it.
singledispatchmakes it easy to extend the central dispatcher
You can attach new dispatchers (and query the registry of existing ones) to a central hub of dispatchers in your code: a common library can expose common dispatchable functions, and each “spoke” that uses the dispatchers can add their own without modifying the original dispatcher code.
singledispatchworks with custom classes and even abstract base classes
Dispatching based on custom classes – including subclasses – is possible, and even encouraged. If you’re using ABCs you can also use them to dispatch to your registered functions.