[Grok-dev] Re: Easier test setup with new grok.testing

[Christian Theune]

Martijn Faassen schrieb:
> Hi there,
> 
> Some feedback to Christian's feedback to help give it some weight:
> 
> Christian Theune wrote:
> [snip]
>> Minor aside: Why do people keep adding the `main` feature? I haven't 
>> used it myself in a long time and my guess is that you actually won't 
>> be able to use it too conveniently because of eggs. (You might be able 
>> to use it when using a custom built interpreter script that you start 
>> this file with. OTOH if you want to run tests selectively the test 
>> runner has much more fine grained options.)
> 
> Agreed, down with the 'main' cruft, let's just use the bin/test 
> testrunner for this and not even mention this cruft in the 
> documentation. It's mostly just cargo cult code people carry over. Get 
> rid of it, please!

w00t.

> [snip]
>> Hmm. Interesting. In my oppinion I shouldn't have to do anything else 
>> for functional or unit tests, so this should stay:
>>
>> import grok
>> test_suite = grok.grok_tests('cave')
> 
> Shorter is indeed better for automation, so let's get this as short as 
> we can.
> 
> Question: what about ftests? You talk about tests, but ftests are run 
> separately by the ftests runner I understand. Is this a misunderstanding 
> and do ftests just work?

The phase we talk about here is *finding* tests. Not running them. 
test_suite needs to be a callable that returns a test suite. The runner 
looks into the suite and finds the various kinds of tests and associates 
them with the layers afterwards. So the test runner makes this 
orthogonal: finding tests and running tests of different kinds.

> Let's talk a bit about control though. 

As you mention control, here is the backwards-compatible way of defining 
the tests:

def test_suite():
     # note the (), this probably needs optimization regarding usability
     suite = grok.grok_tests('cave')()
     suite.addTest(...)
     return suite

 > One thing I do very often with
> doctests is define a bunch of globs to pass in, variables that are 
> available globally. Is there a way to do this with the current code? 
> Ideas? It might be nice if we could state what globs we want to use in 
> the doctest itself. That makes me think we could simply import them:
> 
>   >>> from mypackage.tests import myhelper

For introducing globs we have to make it very easy to create a test 
layer with  setup/teardown methods that can introduce globs.

> That might be actually clearer than the use of globs as it doesn't 
> introduce any special casing at all. Any reason why this wouldn't work?

Actually I like globs, because it makes generic tests reusable. The 
doctests in ZODB for example make it very hard to reuse a generic 
interface test for various implementations because the doctests mention 
a specific implementation for testing. But this might be a usecase that 
we don't want to support.

Christian

-- 
gocept gmbh & co. kg - forsterstrasse 29 - 06112 halle (saale) - germany
www.gocept.com - ct at gocept.com - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development