[Zope-dev] testing everything

[Hanno Schlichting]

On Tue, Aug 18, 2009 at 4:07 PM, Reinout van Rees<reinout at vanrees.org> wrote:
> Question: is there an existing recipe that does this?  Main point is that I
> don't want to specify the dependencies that are to be tested by hand, so that
> seems to rule out z3c.recipe.compattest.  But there's been so much talk about
> testing everything in the last weeks that I thought it better to ask :-)

I wrote http://pypi.python.org/pypi/plone.recipe.alltests, which tries
to do something similar at least.

It's main assumption is to test everything found in the current
working set and let you specify some excludes. A somewhat minimal
example would look something like this:

[alltests]
recipe = plone.recipe.alltests
eggs = ${test:eggs}
exclude =
    zope.foobar
test-script = ${buildout:bin-directory}/test

Where "test" would be a regular zc.recipe.testrunner section.

The integration is currently crude, in that it just calls the existing
test script via os.system instead of doing any direct integration.
That also causes the redundancy of specifying both the eggs and the
test-script. This could be replaced by a pointer to the "test"
section.

The recipe also defaults to running the tests for each package on its
own and has an explicit notion of grouping multiple packages into one
"test group" that is known to run together without side effects. In
Plone testing land the default assumption is, that there is generally
side-effects while testing. For ZTK packages it might be ok to assume
that generally all tests of all packages work without side effects and
only some packages might need to be isolated.

I'd be happy to see patches to the recipe, it's in the collective and
ZPL licensed.

Hanno