[Zope-dev] test setup layer sorting

[Reinout van Rees]

zope/testing/testrunner/runner.py's order_by_bases() function orders
layers by inheritance, basically.  So if you've got a
FunctionalTestLayer that inherits from some BasicTestLayer, the tests
that use the basic test layer will be executed before the functional
test layer ones.

If there's no inheritance, the layers are sorted by name.  Quite sane.

Problem: I use grok. Which does some magic that probably throws off the
code coverage reports.  Quite some code appears untested, even though I
*do* test it.  The problem disappears if just import the python module
first in some basic test before the grokking happened.  I use
z3c.testsetup:

- My basic doctests end up on the
  zope.testing.testrunner.layer.UnitTests layer.

- My functional doctests on the
  z3c.testsetup.functional.layer.DefaultZCMLLayer layer which inherits
  from zope.app.testing's ZCMLLayer.

- zope.app.testing's ZCMLLayer does not inherit from anything, so also
  not from zope.testing's unittest layer.

- 'zope....' is alphabetically behind 'z3c.testsetup....', so the
  functional layer is sorted in front of the basic unittest one.  

- So my grokked code appears untested.

What's the best solution? Should zope.app.testing's ZCMLLayer inherit
from zope.testing's UnitTests layer?  Should z3c.testsetup do some
inheriting of its own?

Reinout


-- 
Reinout van Rees - reinout at vanrees.org - http://reinout.vanrees.org
Software developer at http://www.thehealthagency.com
"Military engineers build missiles. Civil engineers build targets"