After seeing Chris McDonough's excellent paper at the Plone conference on optimizing content delivery using the new IStreamIterator iterface, I began experimenting with implementing such an optimization in ZopePageTemplates. I played around with having 'ZopePageTemplate._exec' request that it receive an iterator, instead of the usual big string. I hoped that such a change might enable greater concurrency and memory footprint, by avoiding creation of the big string at all; instead, medusa to push out the "chunk stream" represented by the StringIO buflist, while the appserver would be free to handle a new request without needing to malloc / copy the data. Here are the timings I have seen so far, using 'zopectl debug', with the following template: --------------- Template source -------------------------------- <html> <body> <div tal:repeat="item python:[x for x in range(1000)]" tal:content="item">ITEM</div> </body> </html> ---------------------------------------------------------------- --------------- Before the patch -------------------------------
Zope.debug('/test_iter', t=1) 250.2 milliseconds Zope.debug('/test_iter', t=1) 106.7 milliseconds Zope.debug('/test_iter', t=1) 106.5 milliseconds Zope.debug('/test_iter', t=1) 124.6 milliseconds
--------------- After the patch --------------------------------
Zope.debug('/test_iter', t=1) 249.2 milliseconds Zope.debug('/test_iter', t=1) 107.2 milliseconds Zope.debug('/test_iter', t=1) 125.0 milliseconds Zope.debug('/test_iter', t=1) 162.1 milliseconds
Given that the performance looks similar in this context, which doesn't benefit from the medusa / concurrency intent of the patch, it seems as though it might be a win (of *course* there aren't any tests for it!) I am attaching the patch I have so far for review and comment. Tres. -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com