[Zope-dev] Test fixture concepts
Tres Seaver
tseaver at palladion.com
Mon Mar 28 10:45:27 EDT 2011
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 03/28/2011 10:04 AM, Jim Fulton wrote:
> On Sun, Mar 27, 2011 at 11:13 AM, Martin Aspeli
> <optilude+lists at gmail.com> wrote:
>> Hi,
>>
>> On 27 March 2011 15:54, Uli Fouquet <uli at gnufix.de> wrote:
>>
>>> The (limited) experiences with py.test, however, were awesome. Some
>>> points that are quite cool IMHO:
>>>
>>> - Easy finding of tests: just write some ``test_function`` in a
>>> ``test_module`` and it will be found and executed. That also makes
>>> py.test tests more readable and maybe more intuitive.
>>
>> I'm not sure this is always a good idea. It feels a bit implicit, and
>> having a base class isn't really a big problem, IMHO. It seems a bit
>> like the kind of thing that sounds cool (look, it's even easier!), but
>> in practice makes little difference.
>
> +1
> +1
> +1
>
> This is especially important for doctests (and manuel) or any
> situation where setup is important and where you can't really guess.
>
>>> - py.test is more widespread in the Python community (that's my
>>> impression; I can't proof it)
>>
>> What about nose?
>
> It looks to me like a layerish mechanism might be possible in nose, or
> at least like
> zope.testing layers could be integrated with nose.
>
>>
>>> - Support of unittest/unittest2: you can write standard lib setups
>>> (defining TestCases; no need to also write testsuite-setup stuff) and
>>> they will be found/executed. zope.testrunner for instance does not
>>> support the new `setUpClass`/`tearDownClass` concept of unittest2
>>> (yes, you would use layers in that case; but it might be nice if
>>> zope.testrunner would support also class-wide fixtures in
>>> unittest2-style; people from other worlds might expect that to work).
>>
>> zope.testing should definitely gain support for the new unittest2
>> hooks. That wouldn't be very hard, though. ;-)
>
> I assume you mean zope.testrunner.
>
>>
>>> Main drawbacks I see on py.test side are:
>>>
>>> - Lack of layer support (yet). Maybe we can do something about that in
>>> `zope.pytest` based on `plone.testing.layer`.
>>>
>>> - Limited doctest support. It is quite difficult (AFAIK) to define
>>> fixtures for doctests or to even set the usual doctest options
>>> (``ELLIPSIS``, ``NORMALIZE_WHITESPACE``, ...) at setup time. Doctests
>>> are simply collected and executed and not much finetuning is possible.
>>
>> With zope.testrunner, you *do* need a test_suite method to run
>> doctests. I think that's a good thing. Look at plone.testing's README
>> for examples.
>
> Again, +1
>
> If I were to use nose or py.tests, I would want to adopt an explicit
> style, which I believe is possible w nose.
FWIW, I thought the same, but haven't missed 'test_suite' appreciably
after a week or so of acclimatization to the nose discovery mechanism.
The only wrinkle I know of is that one doesn't use base classes for test
cases where the base class itself derives from unittest.TestCase:
instead, the base is just used as a "pure mixin".
>> FWIW, I think we should stop using .txt doctests for unit tests.
>
> I disagree, of course.
>
>> Doctests should be used to test *documentation* ("the examples are
>> valid").
>
> Manuel is *much* better for that. (Of course, manuel is arguably a form of
> doctest.)
The Sphinx doctest integration[1] ('make doctest') supports both classic
"interpreter prompt" doctests and a more useful (IMHO) "code-output"
form, such as:
.. testcode::
1+1 # this will give no output!
print 2+2 # this will give output
.. testoutput::
4
[1] http://sphinx.pocoo.org/ext/doctest.html
>> For actual unit tests, writing tests in a unittest class is
>> almost always better in the long run. doctests don't scale well and
>> discourage the kind of ad-hoc "this seems broken, I'll just write a
>> quick test" or "I just fixed a bug, better add a regression test"
>> testing.
>
> You're just not using them correctly. :)
The vast majority of the doctest testcases in zope.* packages fall into
this category: poor isolation, lots of edge cases which would obscure
any real narrative docs, of which there are almost none. I believe the
conflict is intrinsic, here, and not just an accident of careless /
naive implementation: exercising all the preconditions of the contract
of the function-under-test makes for really poor documentation, but is
essential to good testing.
> One thing I hate about unittest is the javiotic ceremony it involves.
> Doctests can cut down on the clutter a lot. I believe that that is
> py.test's strength as well..
"I'll take explicit for $1000, Alex."
> More generally, I'd love to see us adopt another test runner so that
> we can stop maintianing zope.testrunner. When it was written at
> the turn of the century, there weren't good alternatives. Personally,
> I think maintaining it is boring.
Nose and coverage have been pretty good to repoze.* and pyramid_*.
Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 tseaver at palladion.com
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk2QnwcACgkQ+gerLs4ltQ74FQCdGX9twswg6eAy9cnpABAZRrbo
pvoAniGfpBliwnFjqFrV5uqpFcePI2tk
=cqN7
-----END PGP SIGNATURE-----
More information about the Zope-Dev
mailing list