[Grok-dev] grokproject configures z3c.testsetup instead of grok.testing

[Uli Fouquet]

Hi Steve,

thanks also to Kevin for the valuable input!

Steve Schmechel wrote:
> Kevin and Uli,
> 
> Would one of you please enter a bug in launchpad so all these ideas do
>  not get forgotten?  
> 
> I would do it, but I believe the resulting discussion went well beyond
>  my newbie comments on documentation, and a description that I write
>  may not do the ideas presented here any justice.

I guess you could do it very well :-)

But right, there are several problems to think about. Maybe several bug
reports are necessary to reflect this.

Let's see what we have.

> --- On Tue, 3/10/09, Kevin Teague <kevin at bud.ca> wrote:

> > +0.5
> > 
> > I like the idea of aliases within the grok namespace to
> > make imports
> > simpler. But conversely, using imports from the grok
> > namespace tends
> > to
> > convey a deeper integration, it takes a newbie a while to
> > realize,
> > "Oh, Grok is just recommending z3c.testsetup for
> > testing, it's not
> > doing anything magical to z3c.testsetup to make it
> > work."

Yeah, but we have lots of such aliases in Grok, so I don't have too much
problems with it.

> > 
> > > Setting up Layers
> > > -----------------
[snip]
> > 
> > Doesn't z3c.testsetup.doctesting.FunctionalDocTestSetup
> > already look
> > for
> > the packages ftesting.zcml file to use for a default layer?
> > Or am I
> > misunderstanding how layers are setup?

Yes, it does. I am only unsure, whether this is a good policy. If people
have to name their layer (whatever it is), things might become better
understandable.

> > 
> > > In `grokproject` we could provide a locally defined
> > ZCML-layer (as we
> > > already do) which would be included in generated
> > projects and (this time
> > > _explicitly_) be used by local (functional) doctests.
> > >
> > > I admit that this could mean more setup code but
> > people could see how to
> > > setup their own layer from the example code created
> > (and the code would
> > > be really used).
> > >
> > 
> > +1

Okay, then this is bugreport no. 1: Let grokproject explicitly use the
generated layer definition.

[snip: new test markers]

> > > What do you think?
> > >
> > 
> > Well, I've been trying to sort out all of the packages
> > at play
> > (z3c.testsetup, zope.app.testing, zope.testing) and
> > it's making my
> > head spin ...

Right, I can understand it pretty well. It's the same here :-)

> > ... but I think what's confusing me is the Layers. The
> > old-style
> > syntax:
> > 
> >    :test-layer: python
> >    :test-layer: unit
> >    :test-layer: functional
> > 
> > Is confusing because the difference between
> > 'python' and 'unit' tests
> > wasn't the layer (both layers are empty), but how those
> > tests are
> > collected.
> > 
> > The new syntax:
> > 
> >    :doctest:
> >    :functional-zcml-layer: ftesting.zcml
> > 
> >    :doctest:
> >    :layer: sampleapp.tests.FunctionalLayer
> >    :setup: sampleapp.tests.setup
> >    :teardown: sampleapp.tests.teardown
> > 
> >    :unittest:
> > 
> > Is a little simpler in that :unittest: and :doctest: are
> > just
> > informing the
> > test getters to collect tests from that file. But the
> > layers is still
> > confusing.

Well it is less confusing, I hope.

>  I'd rather see the layer names used:
> > 
> >     :doctest:
> >     :layer: FunctionalLayer
> > 
> >     :doctest:
> >     :layer: MyCustomFunctionalLayerName

Yep, that should be possible without too much effort. Philipp already
suggested that to me.

Bugreport #2: Also accept layer names as argument.

> >     :unittest:
> > 
> > Where if the layer is not specified, then one isn't
> > used. But for
> > layer
> > setup: specifying the ZCML file, the setUp() and tearDown()
> > methods,
> > etc.
> > should all happen in the Layer(s) definition. Then in the
> > test cases
> > you
> > simply specify which layer it's part of ...
> > 
> > ... but again, I'm not sure this is possible? And if it
> > is, I think
> > it'd
> > take quite a bit of refactoring in z3c.testsetup.
> > 
> > But it seems to make sense to me: a layer defines a
> > complete set of
> > configuration for setup/teardown.
> > 
> > The confusing bit is that the
> > z3c.testing.testgetter.FunctionalDocTestGetter
> > uses the z3c.testing.doctesting.FunctionalDocTestSetup.
> > This setup
> > then
> > defines it's own layer (with defaults for
> > layer_name='FunctionalLayer',
> > zcml_config='ftesting.zcml'). The ZCML Layer then
> > has a setup with
> > configures
> > the FunctionalTestSetup. But FunctionalDocTestSetup
> > doesn't delegate
> > to
> > the ZCMLLayer setup, but instead specifies in it's own
> > setUp() to use
> > FunctionalTestSetup. Whew!

Right. Congratulations for coming so far. This is also confusing me all
the time. Basically, layers are not very difficult to understand. There
is a class (or some callable) which has a static setUp and tearDown
method and that's it. In other words: regular layers are aggregations of
setup and teardown code. That's where I can follow.

But there are also ZCML layers. They seem not to be ordinary layers, as
their values are stored between single tests and they behave completely
confusing (at least for me). They need a ZCML file (what ordinary layers
do not require), they have 'real' methods (non-classmethods) and store
instance data (also contrary to regular layers). You cannot simply
derive from a ZCML layer (also contrary to regular layers) and write
your own setUp code resulting in a new layer (as you can with ordinary
ones).

Furthermore (as Kevin showed) ZCML layers in functional tests act a bit
like a second layer. I am not sure anymore, whether one should really
call them "layers".

I yet miserably failed to setup custom ZCML layers with custom setup
functions, at least not in a 'regular-layer' fashion.

So, what we might want is a better story for ZCML layers. Maybe we
should even get rid of the name ZCML-*layer* as they behave so different
from regular ones.

> > Purely in pseudo-code, this could support something like
> > this in
> > a Grok-based package's test.py module:
> > 
> > import grok
> > import z3c.testsetup.layers.ZCMLAndZODBDemoStorage # this
> > does not
> > exist!

Yes. Please see my remarks above for the reason. I agree that it would
be nice to have something like this.

So, bugreport #3: Provide some usable default layers with z3c.testsetup,
also for handling ZCML and DemoStorage stuff in an understandable way.

Or more generally: rethink the layer-story especially for ZCML layers.

I think these are the most important problems to tackle. Is there
something more?

Best regards,

-- 
Uli

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Dies ist ein digital signierter Nachrichtenteil
Url : http://mail.zope.org/pipermail/grok-dev/attachments/20090319/051ebb55/attachment.bin