[Grok-dev] grokproject configures z3c.testsetup instead of grok.testing

[Steve Schmechel]

Hi Uli,


--- On Sat, 3/7/09, Uli Fouquet <uli at gnufix.de> wrote:
> Hi Steve,
> 
> Steve Schmechel wrote:
> > Why does the latest grokproject (v1.0a2) when
> installing the latest
> >  grok (v1.0a1) set up a new project's test
> environment following the
> >  documentation that is for grok versions 0.12 and
> older?
> 
> To keep compatible with Grok < 0.13.
> 
> > I know the functionality both packages is similar and
> I realize
> >  grokproject can install older versions of grok, but
> should there be a
> >  limit or should it adjust what it creates based on
> the version
> >  specified?
> 
> It's quite difficult to provide different package
> layouts for different
> Grok versions. Furthermore people could decide after
> creating a project
> to switch the Grok version used. If there would be one
> project layout
> for 0.12 projects and another one for 1.a1, 0.13, etc. we
> would
> definitely run into trouble.
> 
> Currently we support two different project layouts with
> `grokproject`:
> paster and zopectl-based ones. I'd really like to avoid
> to maintain more
> types of projects (say: 0.12-paster, 0.12-zopectl,
> 0.13.-paster,
> 0.13-zopectl, ...).

I certainly wouldn't want that either.

> It would be, however, okay to drop support for certain
> older versions.
> 
> > Observations:
> > 
> > Setup.py has a requirement for z3c.testsetup and the
> initial tests.py
> >  file assigns "test_suite =
> z3c.testsetup.register_all_tests('music')"
> 
> Yes, to keep compatible with Grok < 0.13.

I understand.

> 
> > Following the new documentation in:
> >
> http://grok.zope.org/documentation/how-to/tests-with-grok-testing
> > 
> > it looks like the z3c.testsetup requirement could be
> removed from
> >  setup.py.  (It gets pulled in by some other
> dependency anyway -
> >  probably grok.testing.)
> 
> Yes, when we drop support for older Grok versions. In your
> generated
> grokproject (if it uses Grok >= 0.13) you should be able
> to do that.
> 
> > More importantly, the tests.py stub can be simplified
> to match the
> >  newer documentation.
> > 
> > It goes from:
> > 
> > --Start--
> > import os.path
> > import z3c.testsetup
> > import sample
> > from zope.app.testing.functional import ZCMLLayer
> > ftesting_zcml = os.path.join(
> >     os.path.dirname(sample.__file__),
> 'ftesting.zcml')
> > FunctionalLayer = ZCMLLayer(ftesting_zcml, __name__,
> 'FunctionalLayer',
> >                             allow_teardown=True)
> > test_suite =
> z3c.testsetup.register_all_tests('sample')
> > --End--
> > 
> > To: 
> > 
> > --Start--
> > import grok
> > test_suite =
> grok.testing.register_all_tests('sample')
> > --End--
> > 
> > I don't claim to be an expert in the test
> framework, buildout, or
> >  grokproject so I may be missing something really
> critical here.
> 
> I am not an expert eighter, but the layer defined in the
> former example
> is even not used by default. IIRC it was introduced to have
> some example
> code (for setting up layers) available, as people asked for
> it.
> 
> > It does makes it easier to learn or teach grok when
> the code is simple
> >  and it matches the documentation.
> 
> You're right. I think everybody was a bit busy with all
> the other
> pre-1.0 topics, so thank you for remembering us.
> 
> Nevertheless setting up tests can become complicated and
> then some
> people like to have some more advanced example code in
> their generated
> project. So we have to find a middle course between short
> and
> explanatory code.
> 
> BTW: based on suggestions of Philipp von Weitershausen and
> Sebastien
> Douche kicking my ass (many thanks!) there is a new
> z3c.testsetup
> release (0.3) available, that enables developers to define
> layers, setup
> and teardown methods in each doctest file. An introduction
> can be found
> here:
> 
>   http://pypi.python.org/pypi/z3c.testsetup/0.3
> 
> This release tries to stay compatible with older releases
> but offers
> lots of new stuff.

That is great!  The testing functionality is very important as I try to convince developers who are used to a more "traditional" development style of "it is somebody else's job to tests my code", that they can and should do this themselves as part of their development.

I am all for powerful tools that can be used in a simple way.  (That is what attracted me to Grok in the first place!)  grok.testing looked simpler, but if z3c.testsetup provides more power and can be approached easily, than that is the way to go.

Should we maybe get rid of grok.testing as another option or at least not have it appear to supersede z3c.testsetup in the documentation?

I would rather see one option that works and is explained well than a bunch of choices where I am not sure which is the best.  

We have changed test frameworks at work (csUnit, NUnit, MSTest) and developers have had to redo their tests to support the framework.  (I'm not saying that would happen here.)  Nothing turns developers away from testing more than having it become another maintenance chore.

Thanks again for your great work on this, Uli.

> 
> The most important question, however, is: should we drop
> support for
> Grok < 0.13 with grokproject? IIRC there was also no
> security-fix for
> older versions and it would mainly mean that people using
> older versions
> had to manually do their test setup.
> 

I will leave this discussion to others more knowledgeable than myself.

Regards,
Steve