[Zope-dev] ZPT Optimization Opportunity

Sat Sep 25 22:08:06 EDT 2004

After seeing Chris McDonough's excellent paper at the Plone conference 
on optimizing content delivery using the new IStreamIterator iterface, I 
began experimenting with implementing such an optimization in 
ZopePageTemplates.

I played around with having 'ZopePageTemplate._exec' request that it 
receive an iterator, instead of the usual big string.  I hoped that such 
a change might enable greater concurrency and memory footprint, by 
avoiding creation of the big string at all;  instead,  medusa to push 
out the "chunk stream" represented by the StringIO buflist, while the 
appserver would be free to handle a new request without needing to 
malloc / copy the data.

Here are the timings I have seen so far, using 'zopectl debug', with the 
following template:

--------------- Template source --------------------------------
<html>
<body>
<div tal:repeat="item python:[x for x in range(1000)]"
      tal:content="item">ITEM</div>
</body>
</html>
----------------------------------------------------------------

--------------- Before the patch -------------------------------
>>> Zope.debug('/test_iter', t=1)
250.2 milliseconds
>>> Zope.debug('/test_iter', t=1)
106.7 milliseconds
>>> Zope.debug('/test_iter', t=1)
106.5 milliseconds
>>> Zope.debug('/test_iter', t=1)
124.6 milliseconds
----------------------------------------------------------------

--------------- After the patch --------------------------------
 >>> Zope.debug('/test_iter', t=1)
249.2 milliseconds
 >>> Zope.debug('/test_iter', t=1)
107.2 milliseconds
 >>> Zope.debug('/test_iter', t=1)
125.0 milliseconds
 >>> Zope.debug('/test_iter', t=1)
162.1 milliseconds
----------------------------------------------------------------

Given that the performance looks similar in this context, which doesn't 
benefit from the medusa / concurrency intent of the patch, it seems as 
though it might be a win (of *course* there aren't any tests for it!)

I am attaching the patch I have so far for review and comment.

Tres.
-- 
===============================================================
Tres Seaver                                tseaver at zope.com
Zope Corporation      "Zope Dealers"       http://www.zope.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: zpt_stream_iterator.patch
Type: text/x-patch
Size: 2775 bytes
Desc: not available
Url : http://mail.zope.org/pipermail/zope-dev/attachments/20040925/5b9e984d/zpt_stream_iterator.bin