[Zope] Testing products: Testing all in one batch

[Chris Withers]

Paul Winkler wrote:
>>I have a python script that builds a big command line to test.py of the
>>form:
>>
>>bin/zopectl test Products/Product1|Products/Products2|etc
>>
>>...it's not pretty, but it does work, and lets you exclude geb0rken
>>products like Archetypes from test runs...
> 
> Aha, that's much like what I'm doing except that yours runs in one go
> and thus exposes tests that don't clean up after themselves.
> Which is a Good Thing if I can just find and fix all the nasty things...

No, no, we have several "batches" above, where each batch contains only 
products that don't screw up each others' tests. There were some 
products, mainly from Plone, which we just had to remove from the test 
run altogether, because "THEY SUCK!" ;-)

> Irrelevant to this discussion, but A) it's hardly ancient,

But CMF 1.6 is on the horizon now ;-)

> and B) upgrading to 1.5.x has been postponed indefinitely as it breaks a
> lot of old things (some mine, some third-party) and I have not had any
> time to devote a day or three to resolving the incompatibilities.

Ah, the compatability trap :-S

>>>- Some of my own Product tests run fine in isolation but break when
>>>  run alongside other installed product tests.
>>
>>Then either your tests or the other tests are leaving turds...
> 
> Indeed. I wish it was easier to diagnose and resolve problems like this.

Yup, I know of no other way that goign through case-by-case...

> The problem is that tests seem to be run in alphabetical order, I
> don't know of a way to force another order, 

go and get inside unittest.py and have a play...

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk