[Zope-dev] Test Runner (was Re: Dependencies and future of zope 3)

[Jim Fulton]

On Sep 3, 2008, at 8:37 PM, Tres Seaver wrote:
> The change under discussion about testing is whether or not to remove
> the test requirements as part of 'install_requires' in setup.py.  Some
> folks have objecctions to using the  setuptools-provided  
> 'tests_require'
> field, although I think the argument there is weak:  packages which
> spell 'tests_require' and 'test_suite' in their setup can be tested
> trivially with 'python setup.py test', which seems a win to me.

I agree that this is a win.  It is not a big win IMO, as I prefer a  
separate test runner.

It would be far more useful to me if the data were captured as part of  
egg info.

> I'm not volunteering to write it, but it strikes me as odd that folks
> haven't morphed zc.buildout (and / or associated recipes) to use this
> field.  I *did* write a setuptools add on which saved the
> 'tests_require' info into the EGG_INFO directory:  that package  
> could be
> used to capture the metadata during installation of packages, for
> consumption by a testrunner later.

That's very good.  It would be much much much better if setuptools  
would do this by default. Now that setuptools' development is opening  
up, perhaps we can do this.

The setuptools add-on doesn't help when you don't have source releases  
and many interesting packages get distributed as eggs on Windows.

> I discovered today I think the time is ripe for a "blank buffer"  
> rewrite
> of the testrunner:  it is so full of "twisty passages" that my
> confidence in its own internal correctness is seriously depleted.  It
> has too many features,

I doubt it.  I find the vast majority of it's features very useful. I  
agree that the test discovery is ripe for a revamp.

> and too many incompatible ways to run it:  I
> would love a simpler version, whose discovery logic used egg metadata
> instead of (mal)heuristics (e.g., ordering of '--test-path' and
> '--package-path' arguments can make some tests unfindable).

I'm very open to revamping test discovery.

I'm opposed to a rewrite.  I sure as heck wouldn't be willing to spend  
time on a rewrite. I wouldn't be opposed to replacing out test runner  
with a bunch of nose plugins that added missing features.  I have  
almost zero interest in using setuptools test runner during development.

Jim

--
Jim Fulton
Zope Corporation