[Zope-dev] zope.testrunner and nose count doctests differently

[Chris Withers]

On 03/11/2011 11:05, Jim Fulton wrote:
> On Thu, Nov 3, 2011 at 5:14 AM, Chris Withers<chris at simplistix.co.uk>  wrote:
>> Hi Michael,
>>
>> On 03/11/2011 09:12, Michael Howitz wrote:
>>> Run both test runners with the option -vv to see which tests are run.
>>> (I did this for your code and the list of tests seems to be equal.)
>
> It would be interesting for the rest of us to know the length of that list.

$ bin/test -m testfixtures.tests.test_docs -vv
Running tests at level 1
Running zope.testrunner.layer.UnitTests tests:
   Set up zope.testrunner.layer.UnitTests in 0.000 seconds.
   Running:
  testfixtures/docs/api.txt
  testfixtures/docs/changes.txt
  testfixtures/docs/comparing.txt
  testfixtures/docs/components.txt
  testfixtures/docs/datetime.txt
  testfixtures/docs/description.txt
  testfixtures/docs/development.txt
  testfixtures/docs/exceptions.txt
  testfixtures/docs/files.txt
  testfixtures/docs/index.txt
  testfixtures/docs/installation.txt
  testfixtures/docs/license.txt
  testfixtures/docs/logging.txt
  testfixtures/docs/mocking.txt
  testfixtures/docs/streams.txt
  testfixtures/docs/utilities.txt
   Ran 316 tests with 0 failures and 0 errors in 0.150 seconds.
Tearing down left over layers:
   Tear down zope.testrunner.layer.UnitTests in 0.000 seconds.

$ bin/nosetests testfixtures/tests/test_docs.py -vv
nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$']
testfixtures_nose/docs/api.txt ... ok
testfixtures_nose/docs/changes.txt ... ok
testfixtures_nose/docs/comparing.txt ... ok
testfixtures_nose/docs/components.txt ... ok
testfixtures_nose/docs/datetime.txt ... ok
testfixtures_nose/docs/description.txt ... ok
testfixtures_nose/docs/development.txt ... ok
testfixtures_nose/docs/exceptions.txt ... ok
testfixtures_nose/docs/files.txt ... ok
testfixtures_nose/docs/index.txt ... ok
testfixtures_nose/docs/installation.txt ... ok
testfixtures_nose/docs/license.txt ... ok
testfixtures_nose/docs/logging.txt ... ok
testfixtures_nose/docs/mocking.txt ... ok
testfixtures_nose/docs/streams.txt ... ok
testfixtures_nose/docs/utilities.txt ... ok

----------------------------------------------------------------------
Ran 16 tests in 0.141s

> My guess is that when you're using the zope testruner, you're somehow
> picking up zope.testing.doctest, which counts each doctest example as
> a  test.

The code uses Manuel, under both nose and zope.testrunner:

from doctest import REPORT_NDIFF,ELLIPSIS
from glob import glob
from manuel import doctest,codeblock,capture
from manuel.testing import TestSuite
from os.path import dirname,join,pardir

def test_suite():
     m =  doctest.Manuel(optionflags=REPORT_NDIFF|ELLIPSIS)
     m += codeblock.Manuel()
     m += capture.Manuel()
     return TestSuite(
         m,
         *glob(join(dirname(__file__),pardir,pardir,'docs','*.txt'))
         )

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
             - http://www.simplistix.co.uk