[Zope-dev] test setup layer sorting

[Marius Gedminas]

On Mon, Jul 13, 2009 at 09:27:54AM +0000, Reinout van Rees wrote:
> zope/testing/testrunner/runner.py's order_by_bases() function orders
> layers by inheritance, basically.  So if you've got a
> FunctionalTestLayer that inherits from some BasicTestLayer, the tests
> that use the basic test layer will be executed before the functional
> test layer ones.
> 
> If there's no inheritance, the layers are sorted by name.  Quite sane.
> 
> Problem: I use grok. Which does some magic that probably throws off the
> code coverage reports.  Quite some code appears untested, even though I
> *do* test it.  The problem disappears if just import the python module
> first in some basic test before the grokking happened.  I use
> z3c.testsetup:
> 
> - My basic doctests end up on the
>   zope.testing.testrunner.layer.UnitTests layer.
> 
> - My functional doctests on the
>   z3c.testsetup.functional.layer.DefaultZCMLLayer layer which inherits
>   from zope.app.testing's ZCMLLayer.
> 
> - zope.app.testing's ZCMLLayer does not inherit from anything, so also
>   not from zope.testing's unittest layer.
> 
> - 'zope....' is alphabetically behind 'z3c.testsetup....', so the
>   functional layer is sorted in front of the basic unittest one.  
> 
> - So my grokked code appears untested.
> 
> What's the best solution?

Whatever grok does that interferes with coverage should be fixed.

> Should zope.app.testing's ZCMLLayer inherit
> from zope.testing's UnitTests layer?

Personally I would much prefer for the unit test layer to be sorted
first.  It is already treated specially by zope.testing; I see no harm
in hardcoding its sort order.

> Should z3c.testsetup do some
> inheriting of its own?

That to me sounds like the wrong place to fix this.

Marius Gedminas
-- 
http://pov.lt/ -- Zope 3 consulting and development
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mail.zope.org/pipermail/zope-dev/attachments/20090713/6e19bbb7/attachment.bin