[Zope-dev] Why I dislike narrative doctests

Jim Fulton jim at zope.com
Wed Apr 23 16:39:09 EDT 2008


On Apr 23, 2008, at 4:19 PM, Marius Gedminas wrote:

> Suppose I find a bug (say, zope.testing.testrunner.StartUpFailure
> objects make unittest.TextTestRunner cry).  Being a good developer I
> want to start the bug fix with a unit test.
>
> Now if zope.testing used old-style isolated unit tests, I could open  
> the
> tests.py (or tests/test_testrunner.py) in a text editor and start
> writing:
>
>    class OptionsStub(object):
>        post_mortem = False
>
>    def doctest_StartUpFailure():
>        """Test that StartUpFailure is a proper unittest.TestCase
>
>        StartUpFailure is a TestCase-lookalike that is inserted into  
> the
>        test suite to indicate that a failed import of a test module
>        won't be unnoticed.
>
>>>> from zope.testing.testrunner import StartUpFailure
>>>> options = OptionsStub()
>>>> module_name = 'test_me'
>>>> exc_info = (ImportError, ImportError('zonkulator'), None)
>>>> test = StartUpFailure(options, module_name, exc_info)
>>>> unittest.TextTestRunner(verbosity=2).run(test)
>
>        This is a regression test for http://launchpad.net/bugs/221151
>        """
>
> There, I'm almost done.  Now I can fix the bug and paste the correct
> output into the doctest.
>
> But zope.testing uses narrative doctests instead of isolated unit  
> tests.
> There are 22 plain-text files that together comprise over 4500 lines  
> of
> text.  Now instead of mechanically opening the appropriate test module
> and grepping for the class/function name I have to figure out which of
> the txt files is the appropriate place for the new test.  After that I
> have to figure out how to insert it into the narrative seamlessly.  Or
> decide to create a new file---but a hundred of .txt files each 20- 
> lines
> long is not a good idea, in my opinion.
>
> Suddenly I almost don't want to fix the bug any more.


You don't have to modify any narrative tests.  You can add isolated  
tests to your heart's content.  Just add tests to tests.py. These can  
be isolated doctests or unit tests, although I prefer the former.  I  
haven't done this yet in zope.testing, but I have in lots of other  
projects.

See, for example:

   http://svn.zope.org/zc.buildout/trunk/src/zc/buildout/tests.py?rev=85041&view=auto

I like narrative tests when the tests are reasonably a part of a  
story. Not all tests are, or many tests are there own stories unto  
themselves. In cases like this, small isolated tests are a lot better.

Jim

--
Jim Fulton
Zope Corporation




More information about the Zope-Dev mailing list