[Zope3-dev] Asyncore (and Twisted)

[Itamar Shtull-Trauring]

On Fri, 01 Nov 2002 02:03:27 -0500
Shane Hathaway <shane@zope.com> wrote:

> Well, I don't think networking from threads is the real goal.  What's 
> important is that Zope's ZODB connections are scarce and precious, so 
> time spent in application code must be minimized.  To help achieve
> this, Zope threads must never block for I/O except in rare cases. 
> Therefore all input and output must be fully buffered.

Which is what callFromThread, and then later the transport layer do.

> But when Zope replies to a request with a very large file, the output 
> buffer can be a problem.  A minimal strategy would put the entire file
> 
> into RAM or a disk file before sending it over the wire, which seems 
> wasteful.  It is desirable to start emptying the output buffer as soon
> 
> as Zope starts filling it.  Zope 3 currently does this in the 
> application thread (without blocking), resulting in the earliest 
> possible response to requests.

That's not a problem, you could use Twisted's producer/consumer
interface for transports (buth push and pull producers are supported,
asyncore just does pull). You'd probably want to write one with thread
support that buffers to disk for large amounts of data, similar to the
way Zope3 does buffering.

-- 
Itamar Shtull-Trauring    http://itamarst.org/
Available for Python, Twisted, Zope and Java consulting
***> http://VoteNoWar.org -- vote/donate/volunteer <***