[Grok-dev] grokproject configures z3c.testsetup instead of grok.testing

[Steve Schmechel]

Kevin and Uli,

Would one of you please enter a bug in launchpad so all these ideas do not get forgotten?  

I would do it, but I believe the resulting discussion went well beyond my newbie comments on documentation, and a description that I write may not do the ideas presented here any justice.

Regards,
Steve

--- On Tue, 3/10/09, Kevin Teague <kevin at bud.ca> wrote:

> From: Kevin Teague <kevin at bud.ca>
> Subject: Re: [Grok-dev] grokproject configures z3c.testsetup instead of grok.testing
> To: grok-dev at zope.org
> Date: Tuesday, March 10, 2009, 6:42 PM
> > grok.testing or z3c.testsetup?
> > ------------------------------
> >
> > The main point behind having
> 'grok.testing.register_all_tests' was to
> > have an easier testsetup environment available with
> stock Grok. Import
> > grok and forget about complicately named package
> imports. I think this
> > is still a valid consideration.
> 
> +0.5
> 
> I like the idea of aliases within the grok namespace to
> make imports
> simpler. But conversely, using imports from the grok
> namespace tends
> to
> convey a deeper integration, it takes a newbie a while to
> realize,
> "Oh, Grok is just recommending z3c.testsetup for
> testing, it's not
> doing anything magical to z3c.testsetup to make it
> work."
> 
> > Setting up Layers
> > -----------------
> >
> > Layers are a powerful tool of the Zope testing
> machinery. They were not
> > very well supported with z3c.testsetup < 0.3. You
> could only define one
> > ZCML file or ZCML based layer which was used only for
> functional tests,
> > but all thereof. Furthermore for z3c.testsetup
> 'layer' alway meant
> > 'ZCML-layer' that is setup using
> zope.app.testing.functional although
> > there are also non-ZCML layers and ZCML layers that do
> not need a fully
> > fledged ZODB running in background (okay, the latter
> is rare).
> >
> 
> Right. I've been looking at the 0.2.1 source release
> and "layers"
> seemed
> confusing since it looked like could only have one layer.
> Actually
> allowing for multiple layers in 0.3 makes a lot more sense.
> 
> What about setting up layers with a relational database or
> an LDAP
> instance? A How-To explaining that would be very nice :)
> 
> (or maybe I will write one if I figure it out ...)
> 
> >
> > This raises the question: do we really need a default
> layer? From
> > looking through zope.app.testing I learned, that it
> can be quite
> > cumbersome to look in different locations for
> 'ftesting.zcml' and the
> > results might be unexpected. Sometimes it is like pure
> guessing.
> >
> > From what I learned from developer feedback, I'd
> now tend to opt for
> > skipping the default layer and instead raise an error
> if one is needed
> > but not supplied.
> >
> 
> Doesn't z3c.testsetup.doctesting.FunctionalDocTestSetup
> already look
> for
> the packages ftesting.zcml file to use for a default layer?
> Or am I
> misunderstanding how layers are setup?
> 
> 
> > In `grokproject` we could provide a locally defined
> ZCML-layer (as we
> > already do) which would be included in generated
> projects and (this time
> > _explicitly_) be used by local (functional) doctests.
> >
> > I admit that this could mean more setup code but
> people could see how to
> > setup their own layer from the example code created
> (and the code would
> > be really used).
> >
> 
> +1
> 
> Yes, I think it would make the test setup more obvious if
> the
> boilerplate
> generated by grokproject used an explicit style.
> 
> > And as we are changing everything we could also switch
> to the new
> > testsetup syntax, so that you could setup a functional
> test like this::
> >
> >   My Sample Tests
> >   ===============
> >
> >   :doctest:
> >   :functional-zcml-layer: ftesting.zcml
> >
> >      >>> getRootFolder()
> >      <zope.app.folder.folder.Folder object at
> 0x...>
> >
> > With this registration you could use Kevins ultra
> short setup code::
> >
> >    # tests.py
> >    import grok
> >    test_suite =
> grok.testing.register_all_tests('sampleapp')
> >
> > or you could setup your functional tests like this::
> >
> >   My Sample Tests
> >   ===============
> >
> >   :doctest:
> >   :layer: sampleapp.tests.FunctionalLayer
> >   :setup: sampleapp.tests.setup
> >   :teardown: sampleapp.tests.teardown
> >
> >      >>> getRootFolder()
> >      <zope.app.folder.folder.Folder object at
> 0x...>
> >
> > which would require the complete layer setup beside
> the
> > `FunctionalLayer` definition in tests.py::
> >
> >   # tests.py
> >   import os.path
> >   import grok
> >   import sampleapp
> >   from zope.app.testing.functional import (ZCMLLayer,
> >                                           
> FunctionalTestSetup)
> >
> >   globs =
> z3c.testsetup.functional.doctesting.FunctionalDocTestSetup.globs
> >
> >   def setup(test):
> >       FunctionalTestSetup().setUp()
> >       test.globs = globs
> >       return
> >
> >   def teardown(test):
> >       return FunctionalTestSetup().tearDown()
> >
> >   ftesting_zcml = os.path.join(
> >      os.path.dirname(sampleapp.__file__),
> 'ftesting.zcml')
> >
> >   FunctionalLayer = ZCMLLayer(ftesting_zcml, __name__,
> >                              
> 'FunctionalLayer',
> >                               allow_teardown=True)
> >
> >   test_suite =
> grok.testing.register_all_tests('sampleapp')
> >
> > A bit too much for the beginning, if you ask me (and
> not really needed
> > for the tests done). So I would opt for having a
> default tests.py like
> > the short one above and maybe we should add a link to
> the (then
> > finished) grok testing documentation for people that
> want to create more
> > complicated stuff.
> >
> > What do you think?
> >
> 
> Well, I've been trying to sort out all of the packages
> at play
> (z3c.testsetup, zope.app.testing, zope.testing) and
> it's making my
> head spin ...
> 
> ... but I think what's confusing me is the Layers. The
> old-style
> syntax:
> 
>    :test-layer: python
>    :test-layer: unit
>    :test-layer: functional
> 
> Is confusing because the difference between
> 'python' and 'unit' tests
> wasn't the layer (both layers are empty), but how those
> tests are
> collected.
> 
> The new syntax:
> 
>    :doctest:
>    :functional-zcml-layer: ftesting.zcml
> 
>    :doctest:
>    :layer: sampleapp.tests.FunctionalLayer
>    :setup: sampleapp.tests.setup
>    :teardown: sampleapp.tests.teardown
> 
>    :unittest:
> 
> Is a little simpler in that :unittest: and :doctest: are
> just
> informing the
> test getters to collect tests from that file. But the
> layers is still
> confusing. I'd rather see the layer names used:
> 
>     :doctest:
>     :layer: FunctionalLayer
> 
>     :doctest:
>     :layer: MyCustomFunctionalLayerName
> 
>     :unittest:
> 
> Where if the layer is not specified, then one isn't
> used. But for
> layer
> setup: specifying the ZCML file, the setUp() and tearDown()
> methods,
> etc.
> should all happen in the Layer(s) definition. Then in the
> test cases
> you
> simply specify which layer it's part of ...
> 
> ... but again, I'm not sure this is possible? And if it
> is, I think
> it'd
> take quite a bit of refactoring in z3c.testsetup.
> 
> But it seems to make sense to me: a layer defines a
> complete set of
> configuration for setup/teardown.
> 
> The confusing bit is that the
> z3c.testing.testgetter.FunctionalDocTestGetter
> uses the z3c.testing.doctesting.FunctionalDocTestSetup.
> This setup
> then
> defines it's own layer (with defaults for
> layer_name='FunctionalLayer',
> zcml_config='ftesting.zcml'). The ZCML Layer then
> has a setup with
> configures
> the FunctionalTestSetup. But FunctionalDocTestSetup
> doesn't delegate
> to
> the ZCMLLayer setup, but instead specifies in it's own
> setUp() to use
> FunctionalTestSetup. Whew!
> 
> Purely in pseudo-code, this could support something like
> this in
> a Grok-based package's test.py module:
> 
> import grok
> import z3c.testsetup.layers.ZCMLAndZODBDemoStorage # this
> does not
> exist!
> 
> class SampleAppFunctionalLayer
> (z3c.testsetup.layers.ZCMLAndZODBDemoStorage):
>     layer_name = 'Sample App Functional Layer'
> 
>     def setUp(self):
>         # configures ZCML + ZODB DemoStorage
>         super(SampleAppFunctionalLayer, self).setUp()
>         # additional setup goes here
>         pass
> 
>     def tearDown(self):
>         # tears down ZODB DemoStorage
>         super(SampleAppFunctionalLayer, self).tearDown()
>         # additional tear down goes here
>         pass
> 
> test_suite = grok.testing.register_all_tests(
>     'sampleapp',
>     layers =
> [SampleAppFunctionalLayer(allow_teardown=True), ]
> )
> 
> 
> _______________________________________________
> Grok-dev mailing list
> Grok-dev at zope.org
> http://mail.zope.org/mailman/listinfo/grok-dev