On Mon, Jul 13, 2009 at 12:27:50PM +0000, Reinout van Rees wrote:
On 2009-07-13, Marius Gedminas <marius@gedmin.as> wrote:
Whatever grok does that interferes with coverage should be fixed.
I did some more debugging. In the end it boils down to this:
With z3c.testsetup, you can specify a zcml file at the top of your test file. z3c.testsetup creates a ZCMLLayer with that zcml file.
I my test files, I had two separate zcml files (one registered a bit more than the other). So I ended up with two separate ZCMLLayer subclasses. And both did some grokking. So apparently the teardown of a ZCMLLayer subclass when going from one to the next isn't that clean. There *is* a warning in zope.testing that teardown isn't fully supported.
Ah, that one. zope.testing supports test layers that muck up the global state irrepairably (by letting the layer's tearDown method raise NotImplementedError) and continues running the subsequent test layers in a fresh and squeaky-clean subprocess. Unfortunately, a separate process means separate coverage data tracking, and currently zope.testing doesn't support merging coverage data from several processes. Fortunately, the NotImplementedError in ZCMLLayer's tearDown is just a precaution, 99% of the time it is sufficient to run CleanUp.tearDown to get the global state restored to its pristine condition. You can indicate that it is safe by passing allow_teardown=True to ZCMLLayer's constructor. As a result: * the tests will run marginally faster (no subprocess overhead) * you will be able to use pdb in your tests (zope.testing disables pdb in a subprocess since it wants exclusive control over stdout) * you will be able to get accurate coverage tracing. Now, how you can convince z3c.testsetup to pass allow_teardown=True to the ZCMLLayer it constructs, I don't know. I've never used z3c.testsetup (although it sounds like an interesting library and I should check it out some day). ZCMLLayer's allow_teardown=True badly needs more publicity. Or maybe a ruthless dictator, decreeing that it shall be on by default from some near-future date.
How I solved it: just use one zcml layer in z3c.testsetup and do some extra grokking in the test itself instead of in the second zcml file.
That will work until you decide to introduce a new layer. Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development