[Zope-dev] ZTK automatic discovery with buildbots

[Christophe Combelles]

I've worked on a special buildbot that would be able to create a new ZTK list 
automatically, by testing arbitrary new versions. For this purpose I've created 
z3c.checkversions, and the last version 0.4 has two new options (--blacklist and 
--incremental). http://pypi.python.org/pypi/z3c.checkversions/0.4

I'm using this package in the special following buildbot:
http://buildbot.afpy.org:8018/waterfall

Previously, the buildbot was testing the set of *all the latest* releases, but I 
was never able to get success. So I've changed its behaviour and now it's 
testing new packages one by one, and reduces the version number until tests pass :

1) The first builder on the left uses z3c.checkversions to create a new list 
with *only one* new version, and reject any version found in a blacklist.
2) The 3 builders in the middle do the actual tests against this list. If some 
build fails, the package is added in the blacklist.
3) if tests pass, the version list is kept for the following builds.


The buildbot has started living by itself, and has given the following behaviour :

- it started with ZTK 1.0a2
- it upgraded (arbitrary) zope.dublincore 3.6.0 -> 3.6.3
- tests failed
- it added zope.dublincore 3.6.3 in the blacklist
- it upgraded zope.dublincore 3.6.0 -> 3.6.2
- tests failed
- it added zope.dublincore 3.6.2 in the blacklist
- it upgraded zope.dublincore 3.6.0 -> 3.6.1
- tests failed
- it added zope.dublincore 3.6.1 in the blacklist
- so it keeps zope.dublincore 3.6.0 (until 3.6.4 is released)
- it upgraded lxml 2.2.6 -> 2.2.7
- tests passed
- lxml 2.2.7 is kept for future builds (mmh, ok it wasn't, but it should be :( )
The next run will try another package, and we should be able to get the best 
working ZTK automatically. I still need to do this for all platforms...

This is not optimal, because zope.dublincore 3.6.3 maybe works well when 
upgraded with another package, but I'm not sure how to detect this.

Any idea or suggestion welcome.

Christophe