On Fri, Mar 25, 2011 at 4:24 AM, Wolfgang Schnerring <ws@gocept.com> wrote:
Hello Uli,
I've spent quite some time thinking (and partly coding) about the same issues you mention (but didn't feel ready to talk about it here, yet), so I'm glad that maybe we can start thinking about them together
I think your email addresses two quite different topics, so I'll split my reply. First up: test support for zope.component. Second part: about the concept of test layers.
* Uli Fouquet <uli@gnufix.de> [2011-03-24 01:05]:
Right now we have a problem with pytest integration when it comes to ZCA setups [...] all tests share the same global ZCA registrations and changes to the registrations in one test will affect other tests run thereafter. We have a lack of test isolation.
Exactly. This issue has bitten me too in various places, and as far as I know there are no solutions for it, yet.
The classic solution is to start tests with empty registries, or, if you're using layers, with some baseline registries.
What I envision to solve this issue is that test support for zope.component should work the same way as with the ZODB. There, we have a *stack* of Databases (DemoStorages, to be precise) that wrap each other, where each one delegates reads downwards, but keeps writes for itself. So you might have one database for the layer that provides the baseline, and each test (in its setUp) gets its own database where it can do whatever it wants, because it is thrown away in its tearDown.
In principle, quite a few of the mechanics to do the same thing with zope.component registries are already there (since a registry keeps a list of base registries it will delegate to when something can not be found in itself). And as Hanno and Godefroid mentioned, plone.testing does something in this direction already. (And, it bears repeating, in its core has no dependencies on Plone or Zope2.)
I like the idea of stacking registries.
But as far as I see, there are issues that plone.testing does not address:
1. I've been going over this stuff with my colleague Thomas Lotze, and we realized that just squeezing in a new registry and bending its bases to the previously active one is not enough for full isolation, since this does not cover *deleting* registrations (one, you can only delete the registration from the precise registry it was created in, and two, in the just-bend-the-bases approach, once you delete a registration, it's gone forever).
I think to provide full isolation, we need to make *copies*. And since zope.component in general supports a chain of registries, we probably need to make copies of each of them.
Is deleting registrations important? This seems like an odd use case. If it's needed, I would suggest starting with a baseline (e.g. stack) that doesn't include the component you want to test deleting, then adding in setup.
2. zope.component has two entry points, the global site registry and the current registry (getGlobalSiteManager and getSiteManager). The current registry can be anything, or more precisely, you can call getSiteManager.sethook(callable) and provide a callable that returns the current registry.
I think to provide test support for zope.component (i. e. generally, at the "library level"), we need to support both entry points.
Why? Why would someone care about anything other than the current effective configuration.
The global one is not hard, but the getSiteManager one gets nasty really fast, because of course we have to rely on bending getSiteManager to return the current "test registry"
But as you point out, there's a hook for that.
-- but anyone could at any time call getSiteManager.sethook to change it!
Seriously? Nobody calls that but deep infrastructure code.
Which means we need to intercept that and a) prevent our hook from being replaced and b) inject the new registry into the test stack somehow.
I think you're making this more complicated than it needs to be.
As far as I understand, plone.testing sidesteps these issues by only dealing with the global registry, and specially munging the two known cases in the Zope world where getSiteManager is changed (zope.site and five.something).
**
I'd like to know what people think about this plan. Thomas and I have been over this quite a bit and think it's sound, not overly complicated, and (after we did some experiments) definitely doable. Please do point out stuff we missed. :-)
I think a stack-based approach is very appealing. I think anything more complex is likely to cause more problems than it solves.
I'd very much like to put this functionality into zope.component itself, which of course raises backwards compatibility issues galore,
Not sure why this would have to be backward incompatible, but I'm unconvinced that the complexity comes close to being justified by the benefit.
but any code for this definitely isn't wasted since we can always package it separately if we don't find a way to integrate it.
Thomas and I taken up implementing this, but we can't devote a lot of time to it (about one session per week), so realistically I'm afraid I guess it will take a few months until we have something of substance.
OMG and then who gets to maintain it? Not it! You and Thomas have obviously thought a lot about this and I appreciate the effort you've put into this, but I really don't think it's worth the added complexity. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton