About Dependency Injection
In one of my last posts, I gave an introduction to DI pattern: why it is crucial for software projects & how it differs from Service Locator patterns. Long story short using DI supports Low Coupling & High Cohesion in turn gives us great testability and configurability.
Short overview of available DI libraries in Python
DI is not a commonly used concept in the Python community, actually often it's quite neglected, because "Python is not Java", obviously, but this does not mean that we should not strive to write good, testable code, and that's the reason why we should use DI.
The popularity of Django, which suggests in its documentation a design that does not support any Inversion of Control also does not help with promoting good practices.
Lately, FastAPI gain popularity, and it's the first (to my knowledge) web framework which supports native Dependency Injection
Because of all of these reasons, there are just a few DI containers available, some of them do not even hold the definition of a good container.
What describes a good DI container
A good DI container:
- must not be a Service Locator, and that's easier to get as one might think, just go back to my previous post about this topic
- must not be configurable globally (as a globally available instance), because it introduces problems with the reconfiguration
- must support shared dependencies, so we wouldn't need to exploit the Singleton Pattern
- must support use of profiles, so we can configure it accordingly on different environments
- should support the Decorator Pattern
Overview of available solutions
https://github.com/ivankorobkov/python-inject
disqualified because it uses globally available configuration, and descriptors to pull instances from it, which makes it more of a Service Locator. It also uses Singletons as the default scope which is even worse.
https://github.com/google/pinject
from Google. Feels highly outdated and so it is, as the documentation is still written for Python2. Also, it uses Singletons as default, so even though the container is an instance created "in place" it still returns the same instances again and again. It does not support any reconfiguration solutions for tests.
https://github.com/alecthomas/injector
well... I can't tell anything very bad about it, but nothing good either. My problem with it is its interface, which for me isn't very pleasant to use.
http://python-dependency-injector.ets-labs.org/introduction/di_in_python.html
Very good library, actively supported, it's the only drawback from my point of view is the boilerplate which is necessary to create even a small app with it. But besides that it has many pros: support for multiple types of scopes (default is shared instance, but it includes also very interesting Configuration or Resource); it allows defining small containers, no global configuration, supports modularization; it allows to override the configuration for e.g. tests purpose; provides wire mechanism to integrate with popular frameworks, even Django. A really good choice!
The winner - dependencies aka Injector
My personal winner is https://github.com/proofit404/dependencies which I'm used to cal Injector, because that's the name of the main class.
Fundamentals
- dependencies are matched by
__init__ parameter names
- container returns always a fresh instance when called for one
- it uses shared instance scope by default, and it's the only available scope out of the box.
These simple mechanisms allowed the author of the library to create a really nice abstraction over the configuration of the container and its internals. Actually, everything is very nicely encapsulated, and the user must remember about only these 3 points.
One might ask why parameter names and not types as it usually is in other libraries?
The author justifies it by superior readability of his approach which would not be possible with the usage of types:
from typing import List
class Container:
List = list
Container.List is just a string under the hood. Well, it's really hard to argue with the author, because this design really is pretty, just check it below!
In practice
This is how the code can look like in practice
class CustomerFeedContainer(Injector):
bml_client = BMLAPIClient
drunken_water_info_provider = BMLDrunkenWaterInfoProvider
query_entry = CustomerFeedQueryEntry
GetDrunkenWaterDataForCustomerQuery = GetDrunkenWaterDataForCustomerHandler
@dataclass
class CustomerFeedQueryEntry(InjectorQueryEntry):
GetDrunkenWaterDataForCustomerQuery: GetDrunkenWaterDataForCustomerHandler
@dataclass
class BMLDrunkenWaterInfoProvider:
bml_client: BMLAPIClient
[...]
@dataclass
class GetDrunkenWaterDataForCustomerHandler(
QueryHandler[GetDrunkenWaterDataForCustomerQuery, DrunkenWaterView]
):
drunken_water_info_provider: DrunkenWaterInfoProvider
[...]
CustomerFeedContainer stores the configuration, a call to it's getattr method will return a built instance of one of the defined dependencies. Let's say we want to get an instance of BMLDrunkenWaterInfoProvider which uses BMLAPIClient underneath. We just do:
provider_instance = CustomerFeedContainer.drunken_water_info_provider
what happens under the hood is the match of CustomerFeedContainer.bml_client to BMLDrunkenWaterInfoProvider.__init__(self, bml_client), which allows us to provide different implementations of BMLAPIClient as long as it matches the kwarg name and the interface of course.
Pros
- very easy to use, thanks to it's design the DI configuration is very simple to do
- it allows to override the configuration for tests with the use of
Injector() or previously Injector.let()
- it does introduce to our code any hard dependencies like decorators or descriptors as other libraries usually do. We only need to watch out for override usage in tests, but that can be abstracted out too.
- concrete composition root in the form of
Injector class - supports modularization
- nice documentation
Cons
- its simplicity is also a con, as it does not provide too many features. If you need a Singleton you have to think about how to achieve that, but that's a pro too if you'd ask me - thinking before doing is good
- does not provide any integration for frameworks
- returned instances aren't typed
- only one maintainer, but very active and supportive
Patterns
Composition root
When we do not have a globally stored configuration/container as in one of the fore-mentioned libraries we should strive for small purposeful containers which will be our composition roots for e.g. modules of our application. A good idea is also to separate the business logic dependencies from the infrastructural ones and couple them together somewhere at the root of our application.
class DI(Injector):
db_session = DBSession
class Config(Injector):
db_uri = "postgres://localhost"
DI.db_session
(DI & Config).db_session
class DIAndConfig(DI, Config): ...
DIAndConfig.db_session
too many dependencies anti-pattern
It's not hard to predict that because the container builds always a fresh instance, then we should care about having a small number of dependencies in the classes. What it comes down to in practice is having specific ApplicationServices or as I prefer CommandHandlers which do one thing.
Container encapsulation
As I wrote in one of my last posts injecting the container is a very bad thing to do, because it becomes more of a service locator than a DI solution. However, depending on how our app's architecture is layered, or how we do the assembly of our whole app, for example, based on the framework we're using for web, or when our app is composed of modules we might need to "call" our container from a client class.
My solution to that problem is encapsulating the container behind a nice abstraction:
class Module(ABC):
"""
Module encapsulates a module, providing only 2 actions:
handle - for commands/queries,
override - for overriding module's setup, particularly dependencies.
"""
@abstractmethod
def handle(self, action: Union[Query, Command]) -> Any:
pass
@abstractmethod
def override(self: TMod, **dependencies: Any) -> TMod:
pass
class InjectorModule(Module):
def __init__(self, container: Type[Injector]) -> None:
self.container = container
def handle(self, action: Union[Query, Command]) -> Any:
if isinstance(action, Query):
return cast(InjectorQueryEntry, self.container.query_entry).handle(action)
if isinstance(action, Command):
return cast(InjectorCommandEntry, self.container.command_entry).handle(action)
raise ValueError("Action must be a Command or Query")
def override(self: TInjectorMod, **dependencies: Any) -> TInjectorMod:
return self.__class__(self.container.let(**dependencies))
@classmethod
def new(cls: Type[TInjectorMod], container: Type[Injector]) -> TInjectorMod:
return cls(container)
CustomerActivities = InjectorModule.new(CustomerActivitiesContainer)
That way a Controller element of the app can have direct access only to a Module with a very specific and small interface, and not the whole Injector container.
Encapsulating the container is also crucial from the point of view of its exchange to a different solution someday. It mitigates one of the cons that I mentioned before: it can hide the calls to the override mechanism.
Singletons or long-living instances
Sometimes we might want to use the same instance (or rather have it returned by the Injector) through many calls to the DI container, an example could be the same connection to a database. There are some ways to deal with that.
We can create the instance out of the scope of the container and then pass it as a keyword argument using the override mechanism:
class DI(Injector):
singleton = None
a_singleton = SomeService()
di = DI.let(singleton=a_singleton)
assert di.singleton is di.singleton is a_singleton
Or we can use "the real" Singleton Pattern:
class Singleton(type):
_instances = {}
def __call__(cls, *args, **kwargs):
if cls not in cls._instances:
cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
return cls._instances[cls]
class DBSession(metaclass=Singleton):
def __init__(self, db_uri):
self.db_uri = db_uri
self._is_connected = False
def connect(self):
self._is_connected = True
@property
def is_connected(self):
return self._is_connected
class DI(Injector):
db_session = DBSession
db_uri = "postgres://localhost"
DI.db_session.connect()
assert DI.db_session.is_connected
In one of my projects I used such a nice piece of code to have the UnitOfWork live through several calls to the Module which holds it - to be able to check the emitted DomainEvents (out of scope here)
class FakeInjectorModule(InjectorModule):
"""
A Module that uses the same UoW instance, providing a scoped Singleton
object. This allows to have:
- persistence in FakeRepositories
- checking if UoW was committed
- checking published Events
"""
_unit_of_work: FakeUnitOfWork
def __init__(self, container: Injector) -> None:
self._unit_of_work = container.unit_of_work
self.container = container.let(unit_of_work=self._unit_of_work)
@property
def unit_of_work(self) -> FakeUnitOfWork:
return self._unit_of_work
Of course, I think that building the instance outside of the container is a much better, reliable, and predictable approach than using the Singleton. It's common to forget that we are using a Singleton...
Profiles
To configure dynamically the dependencies, for example, based on environmental variables or something you can use a Factory-like custom provider and @value decorator. Below I used string profiles, but it does not matter where the profiles come from, it can be envs
from dependencies import Injector, this, value
from dataclasses import dataclass
@dataclass
class DB:
name: str
@dataclass
class DBProvider:
profiles: str
def provide(self):
if "dev" in self.profiles:
return DB("DEV DB")
else:
return DB("PROD DB")
class DI(Injector):
db = this.db_provider
@value
def db_provider(profiles: str = ""):
return DBProvider(profiles).provide()
dev_db = DI.db
prod_db = DI.let(profiles="dev").db
Upcoming enhancements
easy Decorator Pattern support
from dependencies import Injector, decorate
class CommandLoggerHandler(CommandHandler):
logger: Logger
handler: CommandHandler
def handle(cmd: Command):
self.logger.log(f"handle command {cmd}")
return self.handler(cmd)
class CustomerActivitiesContainer(Injector):
unit_of_work = SqlAlchemyCustomerActivitiesUnitOfWork
logger = StructuredLogger
command_entry = CustomerActivitiesCommandEntry
query_entry = CustomerActivitiesQueryEntry
ScheduleItemCommand = decorate(CommandLoggerHandler, ScheduleItemHandler)
MarkItemAsDoneCommand = MarkItemAsDoneHandler
GetScheduledItemQuery = decorate(QueryLoggerHandler, GetScheduledItemQueryHandler)
GetAgendaItemsQuery = decorate(QueryLoggerHandler, GetAgendaItemsQueryHandler)
I the future we might get such a nice way to apply the Decorator Pattern. It's supported currently by using this proxy and inner-class Injector containers which makes it very hard to read.
Summary
DI in Python is not a popular subject. We have 3 libraries that match the definition:
- alecthomas/injector I do not like its interface, but I know it's being used in production by some people
- ets-labs/python-dependency-injector very expanded library, with constant support, the problem is it's boilerplate, if that does not bother you, then it's a great choice
- proofit404/dependencies (Injector) simple, but provided all the necessary features. If you need something that's not provided then just think about the design in your application, cause the flow might be somewhere there. Beautiful configuration. Perfect match for agile projects ;)
