Session level scoped fixture being run twice.

[andyDoucette]

Please see this minimal example to reproduce it:

__init__.py:

#Add this directory to the path so we can import things in it directly.
import sys, os; sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))

conftest.py:

from test_a import a_fixture

test_a.py:

import pytest
import os

@pytest.fixture(scope='session')
def a_fixture():
    print('Running a_fixture()...')
    has_run=os.environ.get('FIXTURE_HAS_RUN', 'False')=='True'

    print(f'\n{has_run=}')

    assert not has_run, \
        'a_fixture has already been run it should only run once.'

    # Mark the function as having been run already:
    os.environ['FIXTURE_HAS_RUN']='True'

def test_a(a_fixture):
    pass

test_b.py:

def test_b(a_fixture):
    pass

When I run pytest in this directory, I see that pytest is running the fixture twice, despite the fact that it has a session scope.

[root@opt app]$ pytest --version
pytest 8.3.4


[root@opt app]$ pytest -s --setup-show
============================================== test session starts ==============================================
platform linux -- Python 3.11.2, pytest-8.3.4, pluggy-1.5.0
rootdir: /optHome/app
collected 2 items                                                                                               

test_a.py Running a_fixture()...

has_run=False

SETUP    S a_fixture
        test_a.py::test_a (fixtures used: a_fixture).
test_b.py Running a_fixture()...

has_run=True

SETUP    S a_fixtureE
TEARDOWN S a_fixture
TEARDOWN S a_fixture

==================================================== ERRORS =====================================================
___________________________________________ ERROR at setup of test_b ____________________________________________

    @pytest.fixture(scope='session')
    def a_fixture():
        print('Running a_fixture()...')
        has_run=os.environ.get('FIXTURE_HAS_RUN', 'False')=='True'
    
        print(f'\n{has_run=}')
    
>       assert not has_run, \
            'a_fixture has already been run it should only run once.'
E       AssertionError: a_fixture has already been run it should only run once.
E       assert not True

test_a.py:11: AssertionError
============================================ short test summary info ============================================
ERROR test_b.py::test_b - AssertionError: a_fixture has already been run it should only run once.
========================================== 1 passed, 1 error in 0.01s ===========================================

In my application, this is causing duplicate database entries that are screwing up other parts of the system. Is there a bug in pytest?

[RonnyPfannschmidt]

Each import site is considered a distinct declaration

Please use a conftest

[andyDoucette]

Can you say what you mean by "import site" and "please use a conftest"? I am using a conftest.py file. It's included in the minimal example.

Changing the conftest.py file to this still results in the same behavior:

pytest_plugins= ['test_a']

How would you change the example to make it work?

[andyDoucette]

A quick asside:

I see you changed this to type:question, implying it's not a bug. If we define a bug as "Behavior contrary to some statement in the documentation", which would be my definition (notice how it's user-focused instead of developer-focused), I would say it's a bug because of this documentation:

Scope: sharing fixtures across classes, modules, packages or session
#...
Extending the previous example, we can add a scope="module" parameter to the [@pytest.fixture] (https://docs.pytest.org/en/8.3.x/reference/reference.html#pytest.fixture) invocation to cause a smtp_connection fixture function, responsible to create a connection to a preexisting SMTP server, to only be invoked once per test module (the default is to invoke once per test function). Multiple test functions in a test module will thus each receive the same smtp_connection fixture instance, thus saving time. Possible values for scope are: function, class, module, package or session.

(emphasis mine)

This implies that if we set scope='session', it will run at most once per session. It doesn't say anything about any other special tricks we need to do to get this behavior. I believe this should stay in bug status until the code matches all of the documentation. Note that if the documentation explains a got-ya somewhere else, that's not quite sufficient, since no developer reads the entire documentation before running pytest for the first time. They google something and read the paragraph or two that is relevant to that something.
So, if there is more information needed to use pytest sessions correctly, I would like to see it put in close proximity (or linked to) to that paragraph, in order for other people to be helped too.

[RonnyPfannschmidt]

Pytest considers the place where the fixture is imported as a new definition of a different fixture

So pytest sees one distinct session scope fixture per test module

The recommendation is to use conftest to share fixtures between test modules

[andyDoucette]

I am using conftest.py. can you re-read the thread more closely please? I tried 2 different ways to include the fixture in conftest.py and both resulted in the fixture getting run twice.

…

On Sun, Jan 19, 2025, 5:14 PM Ronny Pfannschmidt ***@***.***> wrote: Pytest considers the place where the fixture is imported as a new definition of a different fixture So pytest sees one distinct session scope fixture per test module The recommendation is to use conftest to share fixtures between test modules — Reply to this email directly, view it on GitHub <#13144 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEVLQ5XGYWORDCDHFGWEOW32LOF2VAVCNFSM6AAAAABVOMSPPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMBQHAZDIMJQGU> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[RonnyPfannschmidt]

Define the fixture only in the conftest

Delete any import of the fixture

[andyDoucette]

I did that. There are no fixture imports and only this in conftest.py: `pytest_plugins= ['test_a']` Same thing. Please try it for yourself with my example and the revised conftest.py. It'll take just 1 minute of your time but then you'll see what I'm talking about.

…

On Mon, Jan 20, 2025, 12:09 AM Ronny Pfannschmidt ***@***.***> wrote: Define the fixture only in the conftest Delete any import of the fixture — Reply to this email directly, view it on GitHub <#13144 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEVLQ5S5B34G7XQ4UZ62MSD2LPWNLAVCNFSM6AAAAABVOMSPPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMBQHE3TKMBYG4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[RonnyPfannschmidt]

Defining a test module as a additional plug-in is utterly wrongly as pytest now has the plugin and the module as fixture source

Delete the fixture from the test module and exactly follow instructions instead of randomly winging in wrong stuff you don't understand

[The-Compiler]

I opened #13148 to track improving the documentation around how to use conftest.py and what consequences importing of fixtures has, as I think we can indeed do a better job there.

As for this issue, your fixture should simply be moved into the conftest.py and not imported in the test module (which indeed will result in pytest seeing two fixtures with the same name).

[andyDoucette]

Thank you @The-Compiler for coming in with a level head at a time of tension. I appreciate the care in your response.

I have some obstacles to doing exactly as you prescribed:

In the way i organize my code, the fixture belongs with the test - it interacts heavily with the test, and the test belongs with the code I'm testing. Therefore I want the fixture to live in the right place or else my code will become much less organized. I also want all of my fixtures to be available globally just in case, something else needs to use them.

If I'm not allowed to import my fixture in the global conftest.py, how can I keep the architecture of my app intact while achieving my objectives? I have about 2-dozen fixtures I'd like available if called upon, and I also have a coding style where I don't permit more than 80 lines of code per file, so "just put the fixtures directly in one file" won't work for me. Do you have any suggestions?

What would it take to change the behavior of pytest to "Just do the right thing" when importing files that have fixtures in them? I'm pretty sure "the right thing" would be the behavior that all python programmers have learned to expect - idempotent imports applying to @pytest.fixture decorated functions just like any other function. The fact that a dozen people have posted issues about this very behavior over the years makes me think there's an opportunity here.

To @RonnyPfannschmidt : After reading up on the 8-12 other related "my session level fixture is being run twice" issues, I can understand if you're annoyed at this topic coming up time and time again. It comes up because the existing way the system works isn't what's expected. Thinking "The users are just doing it wrong" isn't really going to fix the underlying root cause of what's happening. BTW, I didn't randomly come up with the pytest_plugins= ['test_a'] idea. I'm sure I read it in a blog or tutorial at some point. Perhaps that was based on an older version of pytest and things have changed since then. The world is a complicated place. Anyway, thanks for your work on this project. I'm sure it's had a direct or indirect impact on billions of people by now. That's something to be proud of, even if there's always still room for improvement.

[andyDoucette]

Here's a little more detail about my use case:

I'm making an API for a web app. Each route creates a resource using specific other resources as inputs. I do my tests in fixtures, creating the resources as I go and referencing the input resources (fixtures) that are needed to create each new resource. Then I just have dummy test functions that depend on those fixtures. It's proven to be a very elegant way to test all of my code, and basically an indirect way to specify an order between tests (this test must be run before that test, as it depends on it's output). Yes, it breaks the old "each test should be independent" thing, but since the interdependence is merely the existence of a resource, and not it's particulars, it doesn't get too messy.

So, I need a way to be able to organize my app and my fixtures in a directory hierarchy organized by routes while having a single global namespace for fixtures such that any fixture can depend on any other fixture. I currently don't see a way to accomplish this with pytest and I would love your help if you do.

[The-Compiler]

"[The fixture] interacts heavily with the test" but "I want it to be available globally, just in case" seems to me like two things that are opposed to each other.

One thing you could do is placing them into a fixtures_thething.py or somesuch if you have a corresponding test_thething.py, and then declare that in pytest_plugins in a conftest.py.

[andyDoucette]

@The-Compiler: Yeah, global scope (for ease of access) and having a strict package hierarchy are forces pulling in slightly different directions, but it works for now.

Let me see if I understand you right. I already tried a conftest.py with this contents:
pytest_plugins= ['test_a']

where test_a.py had both the test and the fixture in it.

I think you're recommending the exact same thing, but just moving the test to another file that isn't imported anywhere?

[andyDoucette]

I just tried it. Wow, it worked! Now can you help me understand why that worked and why it didn't work when it's the test file that is in the pytest_pulgins? They seem equivalent to me. The same fixture code is added as a plugin (with some additional test code that should be ignored) in both cases and the same test code is run in both cases. I do not understand yet.

But we're half way there already - I am offically unstuck on my project. Thank you so much! :) If I can get the understanding too, then I might not even make the same mistake again (completion).

[The-Compiler]

Before: You're putting your fixture in a place where pytest collects fixtures (your test file) and then importing/loading (pytest_plugins) that a second time from another place where pytest collects fixtures (a conftest.py).

Now: You're putting it in a place which doesn't have a special meaning to pytest, and then loading it from a place where pytest collects fixtures.

[andyDoucette]

Aah!! Thank you so much! 😁 🙏🙏🙏

…

On Tue, Jan 21, 2025, 6:11 PM Florian Bruhin ***@***.***> wrote: Before: You're putting your fixture in a place where pytest collects fixtures (your test file) and then importing/loading (pytest_plugins) that a second time from another place where pytest collects fixtures (a conftest.py). Now: You're putting it in a place which doesn't have a special meaning to pytest, and then loading it from a place where pytest collects fixtures. — Reply to this email directly, view it on GitHub <#13144 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEVLQ5RNUGTKTWGMNOOSV3T2LY57JAVCNFSM6AAAAABVOMSPPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMBUGYZDSMBYGQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>