Fred Drake wrote:
On Mon, 27 Sep 2004 11:25:16 -0400, Tres Seaver <tseaver@zope.com> wrote:
Yes, you read it correctly (although I think the real answer is "identical performance, within margin of error").
With other benefits, that's good enough, too.
Is there anything to indicate that memory usage is actually improved?
I'm not there yet. This is essentially a "zero-copy" optimization; if I can show that the functionality is equivalent, with approximately the same performance under unloaded sites, *then* I can look into whether it helps.
Ok; understood.
I envision two possible benefits, which will materialize only "at scale":
- Removing the need for the appserver thread to malloc and copy the Big String *may* be a win on a memory-constrained system (large allocations can induce funny non-linearities on the underlying malloc implementation). Not having to copy values, even in C, should be a "pure" win, in any case.
Agreed. Doing less is always a win, eventually. Often in combination with doing just a little less in lots of places, but that's ok.
- Returning the iterator to medusa means that the application thread becomes available to service other threads that much more quickly. Even if the malloc issues can't be demonstrated, the increase in concurrency *should* be a win here.
This is interesting. If you're pushing content out the socket earlier,
I'm not looking for that; only to avoid constructing the big result string just to hand off to the publisher. you need to guard against failure much earlier, or be willing
to deal with it in different ways. This is likely less of an issue for sites that have been well-tested and are considered ready for production.
I'll need to review the patch more carefully, especially since I don't know much about the Medusa layer. Does the modified code deal properly with errors encountered late in the transformation of a template?
Transformation is already complete at that point. The only difference is the type of the result returned (eventually) to the publisher. BTW, I looked again at where StringIO is used, and it seems that ZPT constructs extra StringIO objects for (at least some) nested blocks. In that case, we might be able to extend the win by having the "calling" bit use 'list.extend' for its own buflist, instead of calling 'getvalue'. Tres. -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com