[Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Adam GROSZER wrote:

> I'm somewhat vary on unittests. I've seen some damn cryptic ones that
> took a lot of time to decipher.
> A doctest somehow forces you to dump your mind (well at least that, if
> we're not that brilliant techdoc writers).
> OTOH I think most unittests maybe have some comments, worst case they
> don't even use "speaking" variable names.
> I'm not sure whether we can enforce such rules (super thorough, no
> shortcuts-taken, well commented) if we can't easier ones.

I agree that the unit tests themselves often need cleaninng up:

- - They should avoid importing the module-under-test at module scope,
  and minimize any other imports as well:  import errors should cause
  tests to fail, not to be masked.

- - They need to be broken up, so that each testcase method tests exactly
  one code path through each method (or function) under test.  Each
  testcase method should be named to indicate which method/function it
  tests, and which set of preconditions.

- - The local variables need to have *much* better names.

- - They need to avoid using test fixtures from other modules:  having
  to stop to look up exactly what the fixture means or does is a
  "pipeline stall" for the CPU of the programmer tryihg to understand
  the test.

Doing this cleanup is part of the effort I am proposing we undertake:
getting the tests comprehensible is as important as getting to 100%
coverage, because the "lines executed" bit (syntax) that coverage
analyzers report is still not sufficient to ensure that you cover all
the semantics of the module.  Analyzing completeness on the latter
requires understanding what each method or function tested does, and how
each test exercieses it.

I have made a start on such cleanups for zope.interface:

http://svn.zope.org/zope.interface/branches/tseaver-better_unit_tests/

Look particularly at the 'test_interface.py' changes:

http://svn.zope.org/zope.interface/branches/tseaver-better_unit_tests/src/zope/interface/tests/test_interface.py?r1=text&tr1=111109&r2=text&tr2=111110&diff_format=h

and those to 'test_verify.py':

http://svn.zope.org/zope.interface/branches/tseaver-better_unit_tests/src/zope/interface/tests/test_verify.py?r1=text&tr1=111109&r2=text&tr2=111114&diff_format=h

Note as well that I am deliberately leaving off the table the
"integration testing" which is normally done via the zope.testbrowser
layer support:  as with doctests, such tests can be useful, but I would
never rely on them for the "coverage" part of testing a package.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkvMeB0ACgkQ+gerLs4ltQ7rXwCfVxQho7XLjsYTQITxuWHF3/3N
3hcAoLlL/LoaUlLSlaEpjANGPD3jR7rz
=nqY1
-----END PGP SIGNATURE-----