[Zope] ZSERVER, THREADS, PERFORMANCE & FUTURE PERFORMANCE

[Sam Gendler]

>
> I just tested a small python cgi scriptat concurrency level 10:
>
> a cgi script that printed the Content-type and "Hi There!" got ~30 TPS at
>
> a more complicated one that connected to Database and retrieved the
> 20 rows of 700 ran at about ~10 TPS
>
> Just importing DocumentTemplate got it down to 7-8 TPS
>
> Compiling a simple DTML made it go 6.5 TPS
>
> using the DTML it to display the results --> 6.1 TPS

Running the apache bench on my development machine is getting much better results.  I am
just requesting the default index page from zope 2.01, so it is pretty small, but I get
65 TPS whether I use a concurrency of 30 or of 50.  I do, however, still only see 3
python threads executing.  This is a single CPU Mobile Pentium II with 256 MB of memory
with apache and mysql and zope all running, running Mandrake 6.1 with a 2.2.13 kernel.
Another test just ran and I continue to get 65 TPS on my laptop.  Perhaps you are
running an older kernel that doesn't like multiple CPUs, or maybe python doesn't like
SMP regardless of the kernel version?  I will test on my desktop now.  By the way, the
clieet was running on the same machine as the server, presumably eating up much of the
cpu during the test.  When I run it on my 500Mhz P-III desktop with 384MB of RAM, I get
75 TPS running the client on the same machine.  When I run the client on my laptop,
connecting over TCP, I get 77.5 TPS.  This is an admittedly small document, too.
I consistently have 3 python threads in operation, no matter what the concurrency (I was
running 50 concurrent requests, grabbing the doc a total of 10,000 times).  I saw
somewhere that Medusa is touted as being an asynchronous webserver, which should allow
it to run with a minimum of threads, but my experience with such architectures is that
you can usually eke some performance gains out of allowing a small pool of threads to
service incoming transactions (be they http connections or asynchronous callbacks),
especially on a multiple CPU machine.  If it isn't doing this, it is wasting a CPU and
adding overhead every time the process gets switched from one CPU to the other (if linux
does this, I don't know).  Also, it would appear that Zope isn't really behaving in a
multi-threaded manner, either, perhaps some of the guys with digicool addresses in the
CC list would care to comment some on the Zope architecture, as I have not and do not
have time to delve into source code right now.  We should probably move such a
discussion to zope-dev of course.  I am subscribing now.

One last thing.  The zope processes were not utilizing all the memory available to
them.  That is probably because of the small size and limit to the number of objects
being requested.  Each of the 3 zope processes was running at 8MB for the entire length
of the test.

--sam