Hi there, Some of you might have noticed that some time ago the first version of `zope.pytest` was released: http://pypi.python.org/pypi/zope.pytest It's a try to make usage of `pytest` more comfortable in Zope-based environments. `zope.pytest` is mainly based on Martijns, Jan-Wijbrands, and Christian Klingers efforts. I put in some bits too. Right now we have a problem with pytest integration when it comes to ZCA setups: among other things `zope.pytest` offers a pytest-funcarg__ compatible function to automate ZCML based fixtures, i.e. it automatically adds setup and teardown functionality before/after test functions if requested by the test writer like this:: import my.project from zope.pytest import configure from zope.component import queryUtility def pytest_funcarg__config(request): return configure(request, my.project, 'ftesting.zcml') def test_myutil_registered(config): util = queryUtility( my.project.interfaces.ISomeIface, name = 'myutil', default = None) assert util is not None Here for test functions requiring an argument named ``config`` it is guaranteed that before the test function is run a ZCA initialization is performed based on the ZCML configuration given in the ``ftesting.zcml`` of `my.project`. This configuration is cached for the whole test session (the complete test run including all tests). If we assume that the used `ftesting.zcml` provides a named utility named ``myutil`` then the call to ``configure`` in the funcarg__-function should perform that registration and the 'myutil_registered' test should succeed. All that works very well -- in principle. The problem is: the `configure` function calls the registry setup before any real test is run and performs a teardown only after the last of all tests was run. And that leads to side-effects. In other words: all tests share the same global ZCA registrations and changes to the registrations in one test will affect other tests run thereafter. We have a lack of test isolation. Example:: import my.project from zope.pytest import configure from zope.component import queryUtility, provideUtility from zope.interface import Interface def pytest_funcarg__config(request): return configure(request, my.project, 'ftesting.zcml') def test_1(config): util = object() provideUtility(util, provides=Interface, name='a_util') def test_2(config): util = queryUtility(Interface, name='a_util', default=None) assert util is None def test_3(): util = queryUtility(Interface, name='a_util', default=None) assert util is None Here the second test will fail. Even worse, also the third test will fail, which does not require a ZCA setup at all. That's because the global registry is setup before all tests and not torn down before the last test was run. This behaviour is partly intentional: we do not want to setup/teardown the ZCA before/after single test functions as this would increase time consumed by test runs dramatically. Instead we want to reduce the number of ZCA setups/teardown as far as possible. We _could_ do a per testfunction setup/teardown by simply using a different test scope in `configure`. `pytest` offers three kinds of scope for that purpose: `session`, `module`, and `function`. `session` is the current default. Using `function` as scope we would get complete test isolation but pretty long lasting test runs. That could stop people from writing tests. Something we would like to avoid by all means. We could instead use `module` as default scope, so that all test functions inside a module would share one ZCA configuration (and for different test modules the setup would be performed from scratch). This might be a reasonable compromise. Test isolation of single tests inside a test module would of course be broken but one could say: if you want a fresh registry, just put your tests inside a new test module. With Zope test layers there might be a similar problem (each layer can create a global ZCA configuration that then will be shared amongst all applied test methods) but I think people are used to it and can cope with that specific test isolation breakage. A big advantage of test layers over `pytest` testing scopes might be that you can spread your tests associated to a certain layer over many files/modules/packages as you like and the setup/teardown will nevertheless only be performed once for each layer (well, normally at least). If you have one layer and that's enough for you, you will only have one ZCA-setup/teardown in the whole testrun. That's fast and certainly fits many real-world use-cases. Compared to Zope test layers I came to the conclusion that there is not much like this concept already in `pytest` and the behaviour of test layers can not easily be faked. `pytest` provides only the three mentioned scopes as kind of 'natural' layers. Spreading fixtures over many modules (as layers easily do) might contradict with the basic design goals of pytest and I am pretty sure that Holge Krekel wouldn't like it. Overall, we're now looking for a satisfying solution in terms of runtime and isolation. Some of the questions that arise: Would it make sense to bring the Zope layer concept into `pytest`? Are there already possibilities to mimic testlayer-like behaviour in `pytest` which we simply overlooked? Are there cheap/fast ways to cache/restore registry setups we hadn't had on the screen? Really fast setups/cache-restores could make even function-wise registrations a considerable thing. Would it simply be okay to use the 'module' scope for registration setups? Or do you have completely different ideas how to solve that issue? Any comments are really appreciated! Best regards, -- Uli