[Zope-dev] Test runner: layers, subprocesses, and tear down

[Benji York]

On Thu, Jul 3, 2008 at 5:37 PM, Christian Theune <ct at gocept.com> wrote:
> On Thu, 2008-07-03 at 17:22 -0400, Benji York wrote:
>> I'd like to 1) remove the layer tear-down mechanism entirely, and 2)
>> make (almost) all layers run in a subprocess.
>>
>> I want to do #1 because it would simplify the test runner code and no
>> one seems to be using the functionality anyway.  It also appears from
>> reading the code that any tests run in a subprocess (and most are) will
>> never exercise the tear-down mechanism anyway.
>
> +1 in general but -1 on removing the tear down functionality. We use it
> to destroy external databases that where generated for setup.

Ah!  Good point.

>> #2 will add some process start-up overhead, but it'll only be one more
>> process than is already started (and any reasonably-sized test corpus
>> already starts several processes for each test run).  The one exception
>> is for running with -D or with a pdb.set_trace() embedded in the code
>> under test.  For that case we need a switch to say "don't start any
>> subprocesses at all", I suspect that will be spelled -j0.
>
> +1 as well. I'm actually wondering whether we might be able to control
> the pdb through a sub-process.

I don't think it'd be that hard, in general, but the current design of
using stdout and stderr for IPC communication channels is a hindrance.

>> For motivation, some speed comparisons: running a particular test suite
>> with 3876 tests (mostly doctests, and mostly functional) without the
>> patch takes 6 minutes, 42 seconds; my branch runs the same tests in 3
>> minutes and 22 seconds (give or take) on a dual-core box with 3
>> simultaneous subprocesses.
>
> Yay!

I have an 8 core machine that I can't wait to try it on. ;)
-- 
Benji York
Senior Software Engineer
Zope Corporation