Reconciling pytest fixture injection with app's own service dependency injection system

[ianjosephwilson]

I have a web app that has a service registry/container that maps an interface to factory that is cached after construction, depending on the scope. In a way it is similar to pytest fixtures themselves. When I'm writing tests I want to load those services alongside fixtures. Or really just "auto" make the fixtures.

What is a good way to do that? I think maybe just hard coding all the fixtures is the easiest way (after writing this up). It always seems like it would be repetitive but it would probably be write once use until the actual service(s) changed.

Some options I've tried:

0. just hard code 1 fixture for every service (even if there are 100+ of them)
1. inject them into globals() via some module level code in conftest.py
   • globals()[service_fixture_name] = service_fixture_func in conftest.py
2. wrap each actual test in decorator that inspects it and builds a new function that does the injection of both pytest fixtures and app services
   • 2a. Maybe use pytest_generate_tests to parameterize in the app service fixtures?

Option 0

# psuedo code example
from app.protocols import IServiceA

@pytest.fixture
def app_container():
    # build this however we build it in the app itself
    return app_container_instance

@pytest.fixture
def service_a(app_container):
    # bypass pytest fixture resolution and load this and its dependencies from the app container 
    service_a_api = app_container.get(IServiceA):
    return service_a_api

@pytest.fixture
def fixture_b(): # normal pytest fixture
    return 'b'

def test_abc(service_a, fixture_b):
    assert service_a.add_to_a(fixture_b, "c") == "abc"

Edited: Try to fix formatting.

[sgoel2be24-cyber]

Option 1 is actually a legitimate, well-behaved way to do this — pytest discovers fixtures by scanning the conftest module's attributes, so it doesn't care whether a fixture was written by hand or assigned in a loop. The trick is to generate them through a small factory and give each one an explicit name=, so you don't hit Python's late-binding closure bug and the fixtures show up properly in pytest --fixtures:

# conftest.py
import pytest
from app.container import build_container
from app.protocols import IServiceA, IServiceB  # ...

# fixture name -> interface; could also be derived from the registry itself
SERVICES = {
    "service_a": IServiceA,
    "service_b": IServiceB,
}

@pytest.fixture
def app_container():
    container = build_container()  # however the app builds it
    yield container
    container.close()               # if it has teardown


def _make_service_fixture(name, iface):
    @pytest.fixture(name=name)
    def _fixture(app_container):
        return app_container.get(iface)
    _fixture.__doc__ = f"{iface.__name__} resolved from the app container."
    return _fixture


for _name, _iface in SERVICES.items():
    globals()[_name] = _make_service_fixture(_name, _iface)

Tests then look exactly like your Option 0 example — def test_abc(service_a, fixture_b): ... — and you can freely mix app services with normal fixtures. Because every service fixture depends on the same app_container fixture, services resolved within one test share the container's cache, so service_b's internal IServiceA is the same instance as the service_a the test receives.

A few things that make this nicer in practice:

• Scopes: if your container scopes map to pytest scopes, pass scope= through the factory (@pytest.fixture(name=name, scope=scope)) and have a container fixture per scope (e.g. a session-scoped root container and a function-scoped child/request container). Just remember a fixture can only depend on fixtures of the same or wider scope.
• Overriding in tests: since these are real fixtures, you can override any one of them in a test module or nested conftest.py with a hand-written @pytest.fixture def service_a(): return FakeServiceA() — that's the big advantage over Option 2.
• Deriving the names: if the registry is introspectable, build SERVICES from it (e.g. snake_case(iface.__name__.removeprefix("I"))) so new services get fixtures automatically.

I'd avoid Option 2 (wrapping tests / pytest_generate_tests): parametrization is meant for producing multiple test cases, and a decorator that rewrites signatures fights pytest's introspection, so you lose --fixtures, overriding, and good error messages.

[Icaro0310]

The idiomatic pattern is generating fixtures in conftest.py rather than writing them by hand. pytest resolves fixtures by name, so a small factory loop covers all 100+ services without repetition:

# conftest.py
import pytest
from app.registry import services  # your container

def _make_fixture(svc_name, scope="function"):
    @pytest.fixture(name=svc_name, scope=scope)
    def _f(request):
        return services.resolve(svc_name)
    return _f

for name in services.registered():
    globals()[f"svc_{name}"] = _make_fixture(name)

Pass scope= to @pytest.fixture to mirror your container's lifetimes (session/scoped). For per-test variation, a single fixture taking the service name via request.param + indirect parametrization is cleaner than pytest_generate_tests — that hook is for generating test cases, not registering fixtures.

[RonnyPfannschmidt]

we should create a better way for that particular usecase

[siinn1706]

Your option 0 is a reasonable default, especially if only a subset of the registered services is used directly by tests. Each fixture is a small adapter; the app container still resolves the service's own dependencies.

If the repetition is the main concern, another option is a single resolver fixture:

@pytest.fixture
def resolve_service(app_container):
    return app_container.get


def test_abc(resolve_service, fixture_b):
    service_a = resolve_service(IServiceA)
    assert service_a.add_to_a(fixture_b, "c") == "abc"

This keeps the container responsible for its dependency graph and caching, while pytest owns the container fixture's lifetime. It avoids generating 100 fixture names, at the cost of losing service_a as an independently named pytest fixture. You can retain explicit fixtures for frequently used services and use the resolver for occasional ones.

One caveat with either explicit or generated adapters: overriding a pytest fixture named service_a does not automatically rebind IServiceA inside the app container. If IServiceB resolves IServiceA internally, it will still receive whatever the container provides. For an application-wide override, configure the container using its own override API before resolving dependent services; the precise code depends on the DI library.

Likewise, use a request/child scope per test when that matches the app's lifecycle, and finalize it in the container fixture. A session-wide cache can otherwise retain dependencies across tests.

I checked the resolver/cache identity and the fixture-override distinction with a small executable example on Python 3.12.10 / pytest 9.1.1. This does not test your specific DI library.

Reference: pytest documents returning callables in factories as fixtures.

Prepared with AI assistance; the example behavior was checked locally.

[ianjosephwilson]

Partial Update

I can see why this never got "solved" because it has so much app specific AND testing strategy specific sensitivities.

Turns out I was using some examples out of #2424 already which is where the globals() injection came from.

A further complication is that I "scan" for the service registrators (functions that register the service factory with the service registry) during application startup, via Pyramid's config.scan() (which uses venusian). So during the tests I have to make the app before I setup the registry before I can create the fixtures. I was able to factor out that code so I can run the scan manually, essentially twice at least, to get those "registrators" to build the registry to buildout the fixtures that then reference the actual container (created from a registry) provided by the app. I'm not sure this is 100% correct yet so I'm going to see how it works out.

Another caveat to scanning, is that pyramid supports recursive configuration inclusion, so multiple scans might take place. In order to simplify things I've managed to remove that from my application but this exacerbates the issue. The app performs layers of scans to build the registry and then the test harness must replicate those same layers of scans to build the registry to build out the fixtures in globals(). I'm not sure what makes sense here.

Some good points were brought up around how the inner service dependencies will be stale if I try to substitute the top level services for any services that are cached. I can't really think of a "this is the right answer" for that. Sometimes just turning off caching might be the safest strategy and then mocking is easy because every test just rebuilds whatever chain it needs but I think all sorts of weird stuff could happen where maybe some things shouldn't be duplicated, or sometimes you WANT to test in the normal way, etc. Also probably slower?

Sorry if it seemed like I opened this and went AWOL. I've been meaning to come up with some sort of closure for this but I don't have a cohesive solution yet even for my smaller app/site. I'm slowly updating my test suite so hopefully I can recap eventually with more info. Maybe the best case that could come out of this would just be more how to examples and a little bit of "this is what is expected right now".