[Grok-dev] Re: a modest proposal: automatically detected doctests

[Uli Fouquet]

Hi there,

Am Freitag, den 21.12.2007, 17:39 +0000 schrieb Christian Theune:

> On the selection of unit/functional layers I was thinking about using
> those metadata fields, this could be used as a marker for picking it up
> as a docfilesuite either on a (per-grok-package) functional layer or as
> a unit test.
> 
> --snip--
> 
> =========================
> Some feature demonstrated
> =========================
> 
> :Author: The mighty wizard called Tim
> :Test-Layer: (functional|unit)
> 
> ...
> 
> --snap--
> 
> 
> How about that?

In my sandbox I impudently implemented a grok branch that supports
easier setup of doctests in the manner, Christian proposed. 

It provides two new classes in grok.testing::

	`FunctionalTestSetup` and 

	`UnitTestSetup`

that set (hope so) sensible defaults for doctests. 

With this classes setup of tests distributed over a package can be as
easy as this::

   import unittest
   import grok
   import cave # The package that contains the doctest files

   def test_suite():
       setup = grok.testing.UnitTestSetup(cave)
       return setup.getTestSuite()

   if __name__ == '__main__':
       unittest.main(default='test_suite')

This piece of code will grab all relevant unittest files in the `cave`
package (recursivly) and register the contained doctests as unittests.
The accepted files have to meet the following requirements::

Which files are found by default?
---------------------------------

Basically, all files are accepted that

1) reside inside the package passed to the constructor. This includes
   subdirectories.

2) have a filename extension `.txt` or `.rst` (uppercase, lowercase
   etc. does not matter).

3) are *not* located inside a 'hidden' directory (i.e. a directory
   starting with a dot ('.'). Also subdirectories of 'hidden'
   directories are skipped.

4) contain a ReStructured Text meta-marker somewhere, that declares the
   file as a unit test explicitly::

       :Test-Layer: unit

   This means: there *must* be a line like the above one in the
   doctest file. The term might be preceeded or followed by whitspace
   characters (spaces, tabs).

The same is true for setup of functional tests, with the exception, that
the marker text must be:

       :Test-Layer: functional


Customizing your test setup
---------------------------

The most important test setup settings are available and modifyable via
object attributes. 

``UnitTestSetup`` objects support a `optionflags` attribute and provide
a `setUp` and `tearDown` method.

``FunctionalTestSetup`` objects support this and additionally a `layer`
and a `globs` attribute, that stand exactly for what you think and are
allocated with meaningful default values.

Addional test setup keyword parameters can be passed to the constructor
for example like this::

       setup = grok.testing.FunctionalTestSetup(cave, encoding='utf-8')

to parse all doctest files as 'utf-8'.

There is extensive documentation in the test files of the classes in

http://svn.zope.org/Sandbox/ulif/grok-testsetup/src/grok/tests/testsetup/

Nearly everything is modifyable with this test setup classes, namely the
set of accepted filename extensions, the required meta markers etc. You
can filter the set of accepted filenames, dirnames, well, whatever you
like.

Note, that you still have to setup the tests somewhere in your
grokproject manually. But it comes down to writing the few lines shown
above. You can also setup unittests *and* functional tests in one simple
piece of code::

   import unittest
   import grok
   import cave # The package that contains the doctest files

   def test_suite():
       # do test setup
       unit_setup = grok.testing.UnitTestSetup(cave, encoding='utf-8')
       functional_setup = grok.testing.FunctionalTestSetup(cave)
       # create an 'all-in-one' test suite...
       suite = unittest.TestSuite()
       # find and register all doctest files...
       suite.addTest(unit_setup.getTestSuite())
       suite.addTest(functional_setup.getTestSuite())
       return suite

   if __name__ == '__main__':
       unittest.main(default='test_suite')

Could this grok extension be useful?

Kind regards,

-- 
Uli