[Grok-dev] Easier test setup with new grok.testing

[Uli Fouquet]

Hi there,

I created an external branch (Sandbox/ulif/z3c-testsetup) in the
repository for preparation of a z3c.testsetup package, that should
provide the functionality for easier test setups as discussed earlier on
this list.

The work is nearly finished, but I would like to get your input for the
usability of the whole thing before releasing it.

Christian Theune wrote:
> 
> Uli Fouquet schrieb:
> >        import unittest
> >        import grok
> >        import cave # The package that contains the doctest files
> > 
> >        def test_suite():
> >            setup = grok.testing.UnitTestSetup(cave)
> >            return setup.getTestSuite()
> > 
> >        if __name__ == '__main__':
> >            unittest.main(default='test_suite')

> I have the feeling that we could easily reduce this to without major hassle:
> 
> import grok.testing
> test_suite = grok.grok_tests('cave')

> I'm a bit worried that this code is already on the trunk and not on a 
> branch. IMHO it needs a bit more work on the usability side and should 
> be on a branch for now.

Current State
--------------

With the current branch (not 'trunk', this time ;-) it is possible to
register tests like this::

   import z3c.testsetup
   test_suite = z3c.testsetup.register_all_tests('cave')

You can also do::

   import z3c.testsetup
   import cave
   test_suite = z3c.testsetup.register_all_tests(cave)

This registers all tests (unit doctests, functional doctests and
'normal' Python unittests) that provide a special markup string:

  - unit doctest files must contain a string::

	:Test-Layer: unit

  - functional doctest files must contain a string::

        :Test-Layer: functional

  - Python modules with tests must provide a module docstring that 
    contains a line::

        :Test-Layer: python

Doctest files must have .rst or .txt filename extension by default.

'Python tests' must be regular Python modules that should contain
definitions of `unittest.TestCase` classes.

The `register_all_tests` function allows a lot of keyord parameters to
customize finding and setting up of tests, modifying the default
behaviour.

The complete syntax for the function is currently::

     register_all_tests(
         <package_or_package_name_as_dotted_name>,
         [extensions=<filename-extensions-list>,]
         [encoding=<encoding-string>,]
         [checker=<renormalizer-instance-for-functional-doctests>]
         [filter_func=<filter_func-for-all-doctests>,]
           [ffilter_func=<filter_func-for-functional-doctests-only>,]
           [ufilter_func=<filter_func-for-unit-doctests-only>,]
         [fregexp_list=<list-with-regexp-strings>,]
         [uregexp_list=<list-with-regexp-strings>,]
         [setup=<setup-function-for-all-doctests>,]
           [fsetup=<setup-function-for-functional-doctests-only>,]
           [usetup=<setup-function-for-unit-doctests-only>,],
         [teardown=<teardown-function-for-all-doctests>,]
           [fteardown=<teardown-function-for-functional-doctests-only>,]
           [uteardown=<teardown-function-for-unit-doctests-only>,]
         [globs=<dict-with-global-available-things>,]
           [fglobs=<dict-for-functional-doctests-only>,]
           [uglobs=<dict-for-unit-doctests-only>,]
         [optionflags=<optionflags-for-all-doctests>,]
           [foptionflags=<optionflags-for-functional-doctests>,]
           [uoptionflags=<optionflags-for-unit-doctests>,]
         [
          (layer=<ready-to-use-ZCML-layer-for-functional-doctests>,)
          | 
          (zcml_config=<ZCML-config-file-path>,       
           [layer_name=<string>,])
         ]

whereas f... and u... options handle functional and unit doctests
repectively and override any 'global' parameter with the same name. For
example::

  register_all_tests('cave', 
                     extensions=['.foo',],
                     fextensions=['.bar',],
                     uextensions=['.baz',])

will look in .bar-files for functional doctests and in '.baz' files for
unit doctests. .foo-files will be skipped, while

  register_all_tests('cave',
                     extensions=['.foo', '.bar'])

will look in all .bar and .foo files for unitdoctests and functional
doctests.

Beside the package parameter everything is optional. Extensive
documentation to the defaults and everything else is available at::

 http://svn.zope.org/Sandbox/ulif/z3c-testsetup/trunk/src/z3c/testsetup/

Any feedback, how the usability could be improved would be very welcome.

Kind regards,

-- 
Uli