On Mon, Jul 13, 2009 at 09:27:54AM +0000, Reinout van Rees wrote:
zope/testing/testrunner/runner.py's order_by_bases() function orders layers by inheritance, basically. So if you've got a FunctionalTestLayer that inherits from some BasicTestLayer, the tests that use the basic test layer will be executed before the functional test layer ones.
If there's no inheritance, the layers are sorted by name. Quite sane.
Problem: I use grok. Which does some magic that probably throws off the code coverage reports. Quite some code appears untested, even though I *do* test it. The problem disappears if just import the python module first in some basic test before the grokking happened. I use z3c.testsetup:
- My basic doctests end up on the zope.testing.testrunner.layer.UnitTests layer.
- My functional doctests on the z3c.testsetup.functional.layer.DefaultZCMLLayer layer which inherits from zope.app.testing's ZCMLLayer.
- zope.app.testing's ZCMLLayer does not inherit from anything, so also not from zope.testing's unittest layer.
- 'zope....' is alphabetically behind 'z3c.testsetup....', so the functional layer is sorted in front of the basic unittest one.
- So my grokked code appears untested.
What's the best solution?
Whatever grok does that interferes with coverage should be fixed.
Should zope.app.testing's ZCMLLayer inherit from zope.testing's UnitTests layer?
Personally I would much prefer for the unit test layer to be sorted first. It is already treated specially by zope.testing; I see no harm in hardcoding its sort order.
Should z3c.testsetup do some inheriting of its own?
That to me sounds like the wrong place to fix this. Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development