[Zope-dev] IRC discussion about testing

[Jim Fulton]

On Wed, Aug 12, 2009 at 7:07 AM, Wolfgang Schnerring<ws at gocept.com> wrote:
> * Jim Fulton <jim at zope.com> [2009-08-12 11:52]:
>> On Wed, Aug 12, 2009 at 2:31 AM, Wolfgang Schnerring<ws at gocept.com> wrote:
>>>* Jim Fulton <jim at zope.com> [2009-08-12 01:36]:
>>>> In playing with this today, I'm inclined to think that it would be
>>>> simpler to use a list of packages in an option to specify the packages
>>>> to be tested and used a versions section to specify versions, rather
>>>> than using a controlled-packages configuration file.
>>>
>>> This doesn't strike me as simpler to *use* (as I said earlier, I'd much
>>> prefer a *single* gesture of "use this KGS" to set up both the versions
>>> and the list of packages to run tests for, and I think it's worth
>>> spending effort to get there), but I'm not sure that's what you meant.
>>> What makes you prefer two files instead of one?
>>
>> I didn't say anything about 2 files.  I said I prefered a parts list
>> in a single option in combination with a standard buildout versions
>> section.
>
> Sorry for my misunderstanding. In fact, I'm not hung up about the number
> of files all that much, but rather I'm searching for a way not to
> duplicate information. And that seems rather diffcult, since you
> assert...
>
>> - We'll need the versions section to consume the KGS.  That is, given
>> a KGS, you'll often want to use the versions in other buildouts to
>> limit them to the known good configuration.
>
> ...while Martijn said here <http://article.gmane.org/gmane.comp.web.zope.devel/21328>
> that the KGS will need to store more information about each package than
> a versions section can handle.

So there he said:

> I do think we need a computer readable system that includes information
> like this:
>
> * whether a project is in a KGS (such as for the ZTK)
>
> * whether a project is used to test a KGS (a package not in the ZTK but
> used to test whether changes don't induce breakage, let's say,
> grokcore.component)
>
> * the locked down version of the package.
>
> * where the *next* version of the locked down version is being
> developed. Generally this is the SVN trunk, but could be a stable branch.

I'm not sure why this all has to be computer readable.  What is the
use case for computer readable knowledge of whether a package is in
the kgs or just used to test it?  What is the need for computer
readable information about where the package is being developed.


>> - The parts section and versions section could be specified in a
>> single file, so the "gesture" for using them could be pretty simple.
>
> This seems to be the best we can do, given the above requirements.
> If I understand you correctly, that file would then look something like this:
>
> [versions]
> zope.foo = 1.2.3
> grok.bar = 1.1.0
> thirdparty.dependency = 4.4
>
> [zope.foo]
> tested = true
> kgs = ztk
> develop = http://svn.zope.org/repos/main/zope.foo/branches/1.2
>
> [grok.bar]
> tested = true
> develop = http://svn.zope.org/repos/main/grok.bar/trunk
>
> [thirdparty.dependency]
> tested = no
>
> z3c.recipe.compattest/kgs would learn to extract all sections from the
> above where tested=true. And zope.kgs/zope.release could then probably
> be retired (or am I missing something?).

This seems heavier than needed.  Also, if someone extends this,
they're going to get an awful lot of sections that might have names
that conflict with names in their buildout.  I do like the fact that
the versions section is reusable. :)

Here's an alternative:

  [versions]
  zope.foo = 1.2.3
  zope.bar = 1.2.3
  zope.app.baz = 1.2.3
  grok.bar = 1.1.0
  thirdparty.dependency = 4.4

  [ztk]
  projects = zope.foo zope.bar zope.app.baz
  also-tested = grok.bar

Now, if I want to test using this, I do something like:

  [buildout]
  parts = ztk-tests
  extends = the-file-above

  [ztk-tests]
  recipe = our.kgs.testing.recipe
  projects = ${ztk:projects} ${ztk:also-tested}

I think this is much cleaner and gives us:

- Easily reusable information, including versions information that can
  be used by any buildout.

- Information about the projects in the ztk

- Information about additional projects to be tested.

It doesn't give us machine-readable information about where the next
version of the project is being developed.  Why do we need that?

Also, with this, the recipe doesn't have to parse anything.

>> - I think a parts list in an option will be easier to control.  For
>> example, you will be able to use the standard buildout option
>> incrementing and decrementating machinery to modify it.
>
> I don't understand how a parts list could help here.

Now suppose we want to add another project to be tested, 'our.app':

  [buildout]
  parts = ztk-tests
  extends = the-file-above

  [ztk-tests]
  recipe = our.kgs.testing.recipe
  projects = ${ztk:projects} ${ztk:also-tested} our.app

But we don't want to bother testing zope.app.baz. We don't use it and
the tests take too long:

  [buildout]
  parts = ztk-tests
  extends = the-file-above

  [ztk-tests]
  recipe = our.kgs.testing.recipe
  projects = ${ztk:projects} ${ztk:also-tested} our.app

  [ztk]
  projects -= zope.app.baz

Jim

-- 
Jim Fulton