[Zope-dev] coverage.py, profile and hotshot support in Zope's testrunner

[Jim Fulton]

Stefane Fermigier wrote:
> 
> 
> Sidnei da Silva wrote:
> 
>> On Sun, Aug 21, 2005 at 10:20:28AM +0200, Stefane Fermigier wrote:
>> | I have added support for coverage analysis using coverage.py from 
>> Gareth | Rees and Ned Batchelder, as well as support for profiling 
>> using either the | profile or hotshot modules from the Python standard 
>> library, to Zope 2's | test runner test.py.
>> | | Justification:
>> | | - coverage.py will let you interactively focus on the modules you 
>> want to | check for coverage.
>> | | - profile is slow to collect data but quick for analysis.
>> | | - hotshot is fast when collecting but very slow for analysis. On 
>> the other | hand, one can feed the great KCacheGrind tool with hotshot 
>> data after some | transformation and this is the best way I have found 
>> so far for | interactively exploring profile data.
>>
>> Eerm, the test runner in Zope 2 is the same from Zope 3. The version
>> in Zope 3 X3.0 has profile support using hotshot and coverage support
>> using the trace.py module, but the version in Zope 2 is earlier than
>> that.
>>
>> How does that differ from what you're proposing?
> 
> 
> 1. I haven't found hotshot support in either Zope 2.8.1 nor the TRUNK.
> 
> 2. Has I wrote, coverage.py can be used interactively (at least, from 
> the command line) to focus on whichever package you are working on at 
> the moment. You can also collect data from several runs, which is useful 
> for us because we have to test each CPS package in a different run.
> 
> 3. I understand that trace.py and coverage.py have some overlap, and 
> should probably be merged into one great and up to date coverage tool. 
> But before it is done, I find useful to have both tools at our disposal.
> 
> 4. Same for hotshot and profile. Both are useful.

I don't think Sidnei was questioning the value of these, but
just pointing out that another version of the test runner already had
some of this.

I'll note that I'm working on a newer test runner that I hope to use
in Zope 2.9 and 3.2.  The new test runner is a nearly complete rewrite to
provide:

- A more flexible test runner that can be used for a variety of projects.
   The current test runner has been forked for ZODB, Zope 3, and Zope 2.
   That's why the Zope 3 version has features that are lacking in the Zope 2
   version.

- Support for "layers" of tests, so that it can handle unit tests and
   functional tests.

- A slightly better UI.

- Tests (of the test runner itself :)

See:

http://svn.zope.org/zope.testing/trunk/src/zope/testing/testrunner.txt?view=log
http://svn.zope.org/zope.testing/trunk/src/zope/testing/testrunner.py?view=log

I'd like to include coverage and profiling support. (There is partial support
now, but untested.)  Wanna help?

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org