[ZPT] ZPT Performance Twiddling

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I noticed some render-time issues while optimizing a set of pages for a
client, and decided to try creating a cut-down test case.

For background:  my laptop is pretty fast (I've clocked it at 50K+
pystones), and has been able to do around 460 requests/second on a
trivial ZPT page in the root of the ZODB, measured using 'ab -n 1000 -c
4' (ab running on the same machine).

I wanted to see if ZopePageTemplate is measurably slower than "plain"
PageTemplates, but did *not* want to measure security checking (other
than the per-request overhead ZPT incurs to set up the security
context).  I therefore set the template up to render dynamic data from
"simple" Python data structures (lists and dicts).  That overhead (see
the attached spreadsheet) seems to be about 100-125 usec per rendering,
with the difference *dropping* as the page increases in complexity.

A quick summary:

  - A 'null' template (lambda returning a static string), runs in
    0.653 usec.

  - An empty ZPT runs in 168 usec (the equvalent non-Zope PT
    runs in 52 usec).

  - A ZPT which just defines a single name runs in 233 usec (PT,
    115 usec).

  - A ZPT which uses the defined name to fill content of one tag
    runs in 299 usec (PT, 174 usec).

  - The real slowdown comes with 'tal:repeat':  a ZPT which repeats
    over an empty sequence runs in 310 usec, increasing to 1.1 msec
    for a list of 25 (PT, 193 usec - 1 msec).

  - Actually doing replacement inside the loop drives the times up
    (ZPT 312 usec - 3.74 msec, PT 194 usec - 3.62 msec).

  - Nested loops (the case I need to optimize) are pretty bad:
    7 msec for an outer loop of 20 and an inner loop of 2, etc.).
    Each additional iteration (in either loop) seems to cost around
    85 usec.  Note that in this case (20 and 2), the maximum throughput
    on the server (without security or publisher overhead) is ~141
    requests / second, which is fairly pathetic, given how little
    work is actually being done.

I've twiddled with a StringIO replacement whose 'getvalue' returns an
IStreamIterator, rather than the joined string, but don't see any win
there (it comes in slightly slower than the ZPT, but my system isn't
RAM-constrained either).  Such a string buffer might still be a win when
handed off to the publisher, assuming that the iterator's chunks are
of a reasonable size (I haven't tried this yet).

I would really like to have a faster TALInterpreter than the one we have
now, but don't have enough zen in the guts of the shared datastructure
produced by the parser to write one.

Any thoughts?  I'm attaching the script and the spreadsheet with my
results, in case anyone wants to dig further.


Tres.
- --
===================================================================
Tres Seaver          +1 202-558-7113          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDG2tM+gerLs4ltQ4RAribAKDVdOKBgdo3F0MbL4vLEj/o1LmNrQCfQvM9
AESRMs1o6BPGaaS7R32XFtY=
=Z98D
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: perftest_zpt.py
Type: application/x-httpd-cgi
Size: 7411 bytes
Desc: not available
Url : http://mail.zope.org/pipermail/zpt/attachments/20050904/b478a3e1/perftest_zpt-0001.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: zpt_perftest.xls
Type: application/vnd.ms-excel
Size: 18432 bytes
Desc: not available
Url : http://mail.zope.org/pipermail/zpt/attachments/20050904/b478a3e1/zpt_perftest-0001.xls