[Grok-dev] Re: How to hook up custom layers with z3c.testsetup?

[Uli Fouquet]

Hi there,

Philipp von Weitershausen wrote:
> El 20 Jun 2008, a las 18:15 , Uli Fouquet escribió:

Philipp, your mail client is switching the locales ;-)

> > Philipp von Weitershausen wrote:
> >
> >> I was just helping out Ivo on IRC setting up some integration tests  
> >> for
> >> his Grok application and I wondered how you hook up doctests to use a
> >> custom ZCML-based integration layer with the new z3c.testsetup-based
> >> test_suite...
> >>
> >> Uli, any pointers?
> >
> > The layer setup might currently not be as sophisticated as it would be
> > desirable :-(
> >
> > When setting up a layer (only one is supported currently),
> 
> Then what's the point of the :Test-Layer: thing in the doctest? Just  
> to indicate whether you use the (implicitly defined?) ZCML-based layer  
> or not?

I think you might be confused by the marker name 'Test-Layer'. I can
understand that. It should better be named 'Test-Type' or similar.

> Also, I've not managed to make out the difference between the 'python'  
> and the 'unit' setting for this parameter as described by the README.

There are currently three test types supported: (1) unit doctests
('unit'), (2) functional doctests ('functional') and (3) non-doctests
('python'). The latters are registered by `unittest.defaultTestLoader`
and must take care themselves for setup, any functional layers etc. The
z3c.testsetup package currently only cares for the layer setup of the
second type and must confess, that I've never setup a layer for a normal
unit test yet.

The marker 'unit|functional|python' therefore not only decides about
applying or not applying a layer, but also about different test loaders,
different setUp/tearDown-methods, etc.

> I read the docs but I was much confused about all the assumptions the  
> package makes. I think even a medium-sized application will quickly  
> outgrow these assumptions.

That might be right. For me, however, it became handy in several
projects and I think for others too. Anyway, when your testing setup
becomes too complex, you're certainly better off setting up the tests
yourself and in the good old-fashioned way. The package is not a
replacement for any other testing environment but only a 'frontend' that
sets up some defaults. Lots of 'assumptions' (I'd call it 'defaults')
therefore are needed. Without it the package would not make much sense.

For the layer stuff, however, I'd agree that this is not solved
satisfactorily yet. It is quite difficult to setup different layers with
the package. Your proposal below sounds good in that respect.

>  For instance, guessing that
> 
>    :Test-Layer: functional
> 
> *implicitly* loads ftesting.zcml didn't even cross my mind for a  
> second, I thought it was somehow hooked up to FunctionalLayer.

Don't know what you mean here. I might be wrong but when setting up a
functional doctest, there is always some ZCML file involved, isn't it?
The default behaviour of the zope.app.testing package, if I remember
correctly, is to look for such a file somewhere in the depths of
`parts/test/` or so. Compared to this the new default is more
reasonable, I think.

> I think it could be improved in a very simple way: In doctests, you  
> simply refer to the dotted name of the layer, e.g.:
> 
>    :Test-Layer: myapp.tests.IntegrationLayer
> 
> z3c.testsetup just finds the doctests, aggregates them, applies the  
> layer they specify and returns them to the test runner. By default,  
> there'd be a FunctionalLayer in grokproject (like there's already)  
> which is being referred to from the minimal test in app.py.
> 
> WIth this, people can easily create new layers (just create another  
> ZCML file, another ZCMLLayer definition in tests.py) and easily hook  
> them up to their doctests.
> 
> What do you think?

Nice idea!

I agree with the idea in general and personally do not like the tag name
'Test-Layer' to declare a test _type_ at all. 

The problem with it is, that the name is now in use and also its
semantics. We would therefore run some people into terrible trouble when
we change it. What we need here might be a 'migration' policy. What
about the following?

In the beginning let the test types be declared also by another keyword 

  - :Test-Type: unit|functional|python

The keyword :Test-Layer: might be supported for a while, but should
issue deprecation warnings when used.

For the real layer setup I could imagine some (optional) additional
markers (declarations), like::

  - :Test-Layername: NameOfLayer

  - :Test-LayerZCML: filename.zcml  (looked up in package root)

  - :Test-Layerdef: dotted.name.of.definition

So we could move the layer-related parameters step by step from the test
setup file to the real test files, where they might belong. After some
releases then 'Test-Layer' could become a synonym for 'Test-Layerdef'
and the latter might vanish completely some day.

This way one could setup different layers for different tests. Something
I'd love to support.

If others have additional ideas/objections: this is a good time to tell
about it :-)

Kind regards,

-- 
Uli

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: Dies ist ein digital signierter Nachrichtenteil
Url : http://mail.zope.org/pipermail/grok-dev/attachments/20080621/187d485a/attachment.bin