[Zope-dev] IRC discussion about testing

[Jim Fulton]

2009/8/3 Fabio Tranchitella <fabio at tranchitella.it>:
> Hi,

Hi,

I've added zope-dev. I hope you don't mind.

>
> * 2009-08-03 08:59, Christian Theune wrote:
>> Cool! Thanks for helping out! I should be online most of the time and
>> I'd be happy to answer questions.
>
> Tonight I spent some time working on this, and I wrote a mock-up of my idea
> for testing the KGS. I did not want to change anything in compattest, so I
> created a new recipe called z3c.recipe.kgs. Later on we can merge things
> back into compattest.
>
> You can get the recipe from here:
>
> svn://svn.zope.org/repos/main/z3c.recipe.kgs/trunk
>
> There are no tests nor documentation, because it is a mock-up and I did not
> want to spend time on writing documentation for something that maybe is not
> exactly what we need.
>
> Let's say that you are working on transaction and you want to test your
> changes against the KGS, you can simply add the following snippet to your
> buildout.cfg:
>
>  [test-kgs]
>  recipe = z3c.recipe.kgs
>  kgs = http://svn.zope.org/*checkout*/zope.release/trunk/releases/controlled-packages.cfg
>
> You can now run buildout, and you will get ./bin/test-kgs-<package> for
> each package in the KGS, as well as ./bin/test-kgs to run all the tests in
> all the packages in isolated environments.
>
> The KGS is rather big right now, so I suggest to try it with a subset of
> the packages using something like:
>
>  [test-kgs]
>  recipe = z3c.recipe.kgs
>  kgs = http://svn.zope.org/*checkout*/zope.release/trunk/releases/controlled-packages.cfg
>  packages = zope.component
>        zope.interface
>        zope.event
>        zope.configuration
>
> Now.. this simple recipe allows us to test a local change in a package we
> are developing on running the tests of all the packages that are part of a
> KGS. In fact, it reuses the KGS as defined in zope.release, without
> reinventing the wheel, and it is quite generic.
>
> Ideas? Comments? Is this the right direction?

I think this is the right direction. :)

A number of comments below:

- I just tried this out. Unfortunately, I'm getting lots of test
failures using Python 2.5.  That's not the recipe's fault. :)

- A bigger problem is that the runner of all tests eventually dies with
  too many open files on Mac OS X.  Here's the last part of the output:

...
Running test-kgs-zope.app.intid
Running test-kgs-zope.app.keyreference
Running test-kgs-zope.app.layers
Running test-kgs-zope.app.locales
1Running test-kgs-zope.app.localpermission
Running test-kgs-zope.app.locking
Traceback (most recent call last):
  File "bin/test-kgs", line 259, in <module>
  File "/private/tmp/kgs/zc.recipe.kgs/src/z3c/recipe/kgs/runner.py",
line 86, in main
  File "/private/tmp/kgs/zc.recipe.kgs/src/z3c/recipe/kgs/runner.py",
line 33, in start
  File "/usr/local/python/2.5.1/lib/python2.5/subprocess.py", line
593, in __init__
  File "/usr/local/python/2.5.1/lib/python2.5/subprocess.py", line
1002, in _execute_child
OSError: [Errno 24] Too many open files

- The packages from zope.release don't build for me on Windows because
of a dependency on M2Crypto from keas.kmi.  Not a big deal at this
point, since ultimately, we want to use a smaller set.

- The versions specified in the controlled-packages.cfg don't seem to
  be honored except for the package being tested. For example,
  ZODB3-3.9.0b2 is used for test-kgs-ZODB3, but 3.9.0b5 is used for
  everything else.  Fortunately, it appears I can work around this
  using a buildout versions section.

Thanks!

Jim

--
Jim Fulton