[Zope3-dev] Automated compatibility tests

Max M maxm at mxm.dk
Wed Feb 8 07:53:19 EST 2006

Being a long time Product developer for Zope 2, I constantly have a big 
problem with checking compatibility for my products with different 
Python-Zope-Plone-Product combinations.

It is also impossible to guess from the Products on zope.org and 
Plone.org which versions works with what.

As far as I can see, the only reasonable solution to this is to automate 
the task with some kind of testrunner.

I can make this testrunner for my own products. But it would be far 
better if there was a standard way to do it, so that it is possible to 
check different version with each other across developers and platforms.

It could be nice to avoid this same problem in Zope 3, or perhaps even 
to find a way to retrofit it to Zope 2.

If I develop a product to run on Zope 3.2, and my customers are using 
this product, I might not test it on 3.3 or 3.4 later on, as I don't 
really have any reason to. So the product could be broken without 
anybody knowing it before trying it out.

If there where automated compatibility tests it would be possible to see 
when a product was no longer usable.

If I then were to update my product, I would also be able to se if my 
new changes would break compatibility with older versions.

It would also be a way for "core" developers to test if their changes in 
their next Zope version breaks anything in 3. party products. That would 
at least give them a chance to decide to work around it.

The test should include unittest, functional test, migrations of content 
from older product versions etc.

But I imagine that we would need some kind of meta framework for setting 
up tests on a product version per zope version basis.

A result like this would be sweet for everybody:

Zope version    3.2    3.3    3.4
myProduct 1.2           x      x
myProduct 1.1    x      x
myProduct 1.0    x

Does anybody have an idea as to what is necessary to do these automated 
tests, or links to previous efforts in this direction? I imagine 
something like the testrunner that is currently testing out Zope.

Or am I the only one seing this as a problem that should be automated?


hilsen/regards Max M, Denmark

IT's Mad Science

More information about the Zope3-dev mailing list