[Zope] zope, fastcgi, and large files

alt alt@sonic.net
Fri, 5 Apr 2002 21:58:38 -0800


I'm currently using zope 2.3.3, apache 1.3.20, and fastcgi.  So far it
works quite well.  However, I'm trying to support large transfers,
protected via a acl_user (passwords, expiration, etc.).  Each
download/transfer could be 20-50mb.  Initial implementation is done by
setting the response header to be an binary attachment, then use the
RESPONSE.write() command to stream out the data.

However, it appears FCGIServer buffers the *entire* 20mb before
sending it out to the apache mod_fastcgi.  This won't work for me, as
it's a huge memory hog, plus there's a noticeable delay before the
buffer is flushed out to apache.

The FCGIChannel sendDataRecord() and push() shows that buffering is
turned on, due to some thread related issue.  Forcing it off indeed
shows problems when there are multiple connections

If I didn't have to deal with passwords/expirations, I could have done
this outside of zope.  

Is there a better way to handle this?  How does www.zope.org deal with
this?

Thanks,
Albert Ting (alt@sonic.net)