[Zope-dev] RFC: adding support for testing from bare setuptools

[Marius Gedminas]

On Wed, Sep 17, 2008 at 12:52:49PM -0400, Tres Seaver wrote:
> Proposal
> --------
> 
> Instead, what I *am* proposing is adding metadata which allows consumers
> of such packages to verify that the package is downloaded / installed
> correctly.  In particular, I want for the following scenario to work
> seamlessly::
> 
>  $ /path/to/virtualenv --no-site-packges sandbox
>  ...
>  $ cd sandbox
>  $ tar xzf zope.foo.bar-3.5.6.tar.gz
>  $ cd zope.foo.bar-3.5.6           # [1]
> 
>  $ ../bin/python setup.py test     # [2]
>  # Runs egg_info, installs regular and testing dependencies, and
>  # runs all unit (non-layer) tests

I don't like the idea that running the tests installs additional
packages into my environment without me explicitly asking for it.

>  $ ../bin/python setup.py test    # [3]
>  # Runs egg_info, installs regular and t

This is truncated.

>  # runs *all* tests

...
> [3] In the second method, we use the setuptools entry point mechanism to
> register a custom 'test' command[5].  This is a wee bit fragile, as
> running 'setup.py test' without first generating the egg_info will use
> the "standard" testrunner (but only the first time).  If such fragility
> is an issue, we can register a separate command, e.g. 'ftest';  in
> either case, we have to document that you need to run 'setup.py
> egg_info' one time before running the custom command.

(The more I learn about setuptools, the less I like them.)

> If the ideal
> installation outlined in [1] above worked, that would solve the problem,
> too, because the egg-info would be generated by easy_install.
> 
> This techniquie drives the zope.testing.testrunner via an entry point,
> which is cool, but it cannot expose all of testrunner's features.
> In particular, the configuration knobs which the testrunner expects to
> get on the command-line clash with distutils' own command-line syntax
> (yes, I've tried to make it work).  The command I have provided just
> runs all the tests, unit and functional, which seems to be reasonable
> assurance for the package conumer that the package installed cleanly.

*nod*

> Implementation
> --------------
> 
> The intent is to add the attached module as
> 'zope.testing.testrunner.eggsupport',

At this point I'm wondering whether we shouldn't split out the
testrunner into a separate zope.testrunner package.

> and then to modify our packages
> (opportunistically, or through a 'geddon) as follows:
...
> The second attachment is a diff showing the changes to a typical Zope2
> product's setup.py to support this proposal.

I'm +0.5 for the idea, although the amount of ritual dancing in the
setup.py seems a bit excessive.

Marius Gedminas
-- 
99 little bugs in the code,
99 bugs in the code,
fix one bug, compile it again...
101 little bugs in the code....
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mail.zope.org/pipermail/zope-dev/attachments/20080917/584f6686/attachment.bin