[Zope-dev] Testrunner option for running tests in random order?!?

[Christian Theune]

On 10/19/2009 11:57 AM, Jonathan Ballet wrote:
> Hello there,
> 
> as I mentioned by Sebastien, the patch has been commited into the
> sdouche-shuffle branch. It now has tests and a bit of documentation
> too.
> 
> Regarding CLI arguments, I'm not sure this could be changed like
> Christian proposed. If you know how to do it, I'm listening, 'cause I
> don't know! :)
> 
> I have one more thing I would like to discuss: how should this feature
> behave with the --repeat feature?
> Currently, the shuffling is done before the repeat feature. This means
> that tests will *not* get reordered for each loop (just once, at the
> very beginning of tests launching).
> I know how to do it, so that for each loop, tests will get a different
> order (basically, I would propose to wrap the __iter__() method of the
> TestSuite used to launch tests, and to shuffle here):
> 
>     class ShuffledTestSuite(unittest.TestSuite):
>         def __iter__(self):
>             tests = list(super(Shuffled, self))
>             seed = long(time.time() * 256)
>             shuffler = random.Random()
>             shuffler.shuffle(tests)
>             return iter(tests)
> 
> However, using this means we would loose the seed printing (which
> seems very convenient to me). I'm not sure if we could get it and
> print it for each loop...

I'd say leave it as is for now. I think it's more important to be able
to seed (for the sake of reproduction, so btw: do you log the seed if
you generated one?) externally than to reorder during each repeated run.

Also, we can add that later on in case you figure out a way how to do it. ;)

Christia

-- 
Christian Theune · ct at gocept.com
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1
Zope and Plone consulting and development