[Zope-dev] Testrunner option for running tests in random order?!?

[Jonathan Ballet]

Hello there,

as I mentioned by Sebastien, the patch has been commited into the
sdouche-shuffle branch. It now has tests and a bit of documentation
too.

Regarding CLI arguments, I'm not sure this could be changed like
Christian proposed. If you know how to do it, I'm listening, 'cause I
don't know! :)

I have one more thing I would like to discuss: how should this feature
behave with the --repeat feature?
Currently, the shuffling is done before the repeat feature. This means
that tests will *not* get reordered for each loop (just once, at the
very beginning of tests launching).
I know how to do it, so that for each loop, tests will get a different
order (basically, I would propose to wrap the __iter__() method of the
TestSuite used to launch tests, and to shuffle here):

    class ShuffledTestSuite(unittest.TestSuite):
        def __iter__(self):
            tests = list(super(Shuffled, self))
            seed = long(time.time() * 256)
            shuffler = random.Random()
            shuffler.shuffle(tests)
            return iter(tests)

However, using this means we would loose the seed printing (which
seems very convenient to me). I'm not sure if we could get it and
print it for each loop...


Reviews, comments and feedbacks are welcome!
Cheers,

 Jonathan