[Zope-dev] Re: Test runner: layers, subprocesses, and tear down
Benji York
benji at zope.com
Fri Jul 4 11:50:34 EDT 2008
On Thu, Jul 3, 2008 at 5:22 PM, Benji York <benji at zope.com> wrote:
> I'm working on making the zope.testing test runner run tests in
> parallelized subprocesses. The option will likely be spelled -j N,
> where N is the maximum number of processes.
The branch (svn+ssh://svn.zope.org/repos/main/zope.testing/branches/benji-parallelize-subprocesses)
is feature complete. I basically did a very simple transformation that
resulted in the runner spawning subprocesses in threads, several at a
time, instead of spawning them serially. The patch is less than 250
lines. Any critiques of the changes would be appreciated.
No changes to the tear-down mechanism were made. All existing tests
pass without modification. There aren't yet any tests for the new
functionality. It may be tricky to test, so I have to think about that
bit.
I found that to eliminate nearly all CPU idle time, I had to use -j4 on
my two core laptop.
For a particular test corpus on a 4 core machine -j1 (the default) takes
about 7 minutes -j6 takes about 2 minutes 20 seconds.
If you use zc.buildout, then you can try the branch by checking it out,
adding a "develop" entry into your buildout config referencing it, and
updating any version spec for zope.testing to "3.6dev". I'd really like
third-party confirmation of the total test time reductions I've seen.
--
Benji York
Senior Software Engineer
Zope Corporation
More information about the Zope-Dev
mailing list