[Zope-dev] test setup layer sorting

[Marius Gedminas]

On Mon, Jul 13, 2009 at 12:27:50PM +0000, Reinout van Rees wrote:
> On 2009-07-13, Marius Gedminas <marius at gedmin.as> wrote:
> >
> > Whatever grok does that interferes with coverage should be fixed.
> 
> I did some more debugging.  In the end it boils down to this:
> 
> With z3c.testsetup, you can specify a zcml file at the top of your
> test file.  z3c.testsetup creates a ZCMLLayer with that zcml file.
> 
> I my test files, I had two separate zcml files (one registered a bit
> more than the other).  So I ended up with two separate ZCMLLayer
> subclasses.  And both did some grokking.  So apparently the teardown
> of a ZCMLLayer subclass when going from one to the next isn't that
> clean.  There *is* a warning in zope.testing that teardown isn't fully
> supported.

Ah, that one.

zope.testing supports test layers that muck up the global state
irrepairably (by letting the layer's tearDown method raise
NotImplementedError) and continues running the subsequent test layers in
a fresh and squeaky-clean subprocess.

Unfortunately, a separate process means separate coverage data tracking,
and currently zope.testing doesn't support merging coverage data from
several processes.

Fortunately, the NotImplementedError in ZCMLLayer's tearDown is just a
precaution, 99% of the time it is sufficient to run CleanUp.tearDown
to get the global state restored to its pristine condition.  You can
indicate that it is safe by passing allow_teardown=True to ZCMLLayer's
constructor.  As a result:

  * the tests will run marginally faster (no subprocess overhead)

  * you will be able to use pdb in your tests (zope.testing disables pdb
    in a subprocess since it wants exclusive control over stdout)

  * you will be able to get accurate coverage tracing.

Now, how you can convince z3c.testsetup to pass allow_teardown=True to
the ZCMLLayer it constructs, I don't know.  I've never used
z3c.testsetup (although it sounds like an interesting library and I
should check it out some day).

ZCMLLayer's allow_teardown=True badly needs more publicity.  Or maybe a
ruthless dictator, decreeing that it shall be on by default from some
near-future date.

> How I solved it: just use one zcml layer in z3c.testsetup and do some
> extra grokking in the test itself instead of in the second zcml file.

That will work until you decide to introduce a new layer.

Marius Gedminas
-- 
http://pov.lt/ -- Zope 3 consulting and development
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mail.zope.org/pipermail/zope-dev/attachments/20090713/d4061f0d/attachment.bin