[Grok-dev] Easier test setup with new grok.testing

[Uli Fouquet]

Hi there,

Christian Theune wrote:

> > I just merged my testsetup extension into the branch, because I am lazy,
> > my memory is like a sieve and I didn't want to remember all the dirty
> > details of test setups any more when starting a new grok project. Hope,
> > nothing will break.
> 
> With "the branch" you mean "the trunk", right? At least that's where 
> your links go.

Sorry, yes, the trunk was meant.

> > To make use of this extension in your grok project, you have to perform
> > three steps:
> > 
> > 1) Mark your doctests with either::
> > 
> >      :Test-Layer: functional
> > 
> >    or::
> > 
> >      :Test-Layer: unit
> > 
> >    Choose the first term, if your file contains functional tests, the 
> >    latter term, if it contains unit tests.
> 
> Nice.

Hope you like it. It was your idea ;-)

> >        import unittest
> >        import grok
> >        import cave # The package that contains the doctest files
> > 
> >        def test_suite():
> >            setup = grok.testing.UnitTestSetup(cave)
> >            return setup.getTestSuite()
> > 
> >        if __name__ == '__main__':
> >            unittest.main(default='test_suite')
> 
> I have the feeling that we could easily reduce this to without major hassle:
> 
> import grok.testing
> test_suite = grok.grok_tests('cave')

I thought about some convenience functions before, too. Something like::

	grok.register_unit_doctests(cave) # or ...('cave')
	grok.register_functional_doctests(cave) # or ...('cave')
        grok.register_all_doctests(cave) # or ...('cave')

but were not sure, whether this would be too much magic. You loose at
least the finegrained control of the test setups, but, yes, I personally
would like to see this available as well and would be glad to implement
it.

Actually, I already did something like this locally :-)

The name `grok_tests` might be misleading IMHO, because the tests are
'registered' and not grokked and only doctests are handled currently.

> Minor aside: Why do people keep adding the `main` feature? I haven't 
> used it myself in a long time and my guess is that you actually won't be 
> able to use it too conveniently because of eggs. (You might be able to 
> use it when using a custom built interpreter script that you start this 
> file with. OTOH if you want to run tests selectively the test runner has 
> much more fine grained options.)

Just used it, because I'm used to do so ;-)


> >    You want to setup unittests and functional tests once in a row? Here 
> >    we go::
> > 
[snip: registering unit and functional tests at once]
> 
> Hmm. Interesting. In my oppinion I shouldn't have to do anything else 
> for functional or unit tests, so this should stay:
> 
> import grok
> test_suite = grok.grok_tests('cave')
> 

> I'm sorry that I'm stepping in only every now and then, I'm trying to 
> become a bit more regular again. Hopefully I didn't miss any important 
> discussion when throwing things in here now.

No worries! Thanks for the input!

> I'm a bit worried that this code is already on the trunk and not on a 
> branch. IMHO it needs a bit more work on the usability side and should 
> be on a branch for now.

It was out in a branch for weeks and I asked for reviews before. Okay,
it was not grok/branches but my sandbox, but announced in public and
reviewable for every reader of this list or user of Google. The code is
tested and seems not to break things (I tried to be very careful about
that). 

Furthermore, your suggestions, if I understand it correctly, do not
touch the general functionality but describe an extension of the already
implemented stuff. So further changes should not break any app.

I would be glad to implement your suggestions, if we can agree on a
certain syntax for the 'catch-all' registerer. My proposal would be::

	grok.register_doctests(pkg|'pkgname')

maybe with a couple of optional arguments like::

	grok.register_doctests(pkg|'pkgname'[, 
                               extensions=['.rst','.txt']])

Last not least I am pretty sure, the code would remain unnoticed in a
dark, hidden and rotten corner of the repository if I would not have
merged it yesterday. ;-)

One thing I was thinking about beside this: The code is pretty
independent from grok 'core' functionality and *might* be usable for
non-grok projects as well. I think it's nice to have it with every
installed grok package, but it may be of better use if we put it in a
separate package named `z3c.testsetup` or similar.

In this case it would indeed have been a failure to check this into the
trunk.

For now thanks to everybody for valuable input!

Kind regards,

-- 
Uli