[Zope-dev] Serving large files

[Chris McDonough]

Oops.. should have read further on there...

On Sep 18, 2006, at 4:44 PM, Chris McDonough wrote:

> Could you return an iterator to the server that knows enough to  
> open its own database connection?  Provide it with a callback that  
> opens the connection and finds and chunks the relevant object?
>
> - C
>
> On Sep 17, 2006, at 3:09 PM, Sidnei da Silva wrote:
>
>> I remember having a conversation with Jim at some point where he
>> proposed a strategy for requests that could potentially take a long
>> time to finish. If I recall correctly, he proposed having a separate
>> ZODB connection pool.
>>
>> One thing that is problematic today is serving large files
>> from the ZODB (ignoring the upcoming blob support).
>>
>> a) You can't return an iterator that reads from the ZODB, because by
>>    the time the iterator is consumed the connection has already been
>>    closed. I believe the iterator is not consumed in the same thread,
>>    which can cause yet more issues.
>>
>> b) If you dump the data to a temp file and return that as a file
>>    iterator, it can potentially take twice the time.
>>
>> c) If you use RESPONSE.write() you can break other
>>    applications. ExternalEditor comes to mind.
>>
>> d) If you just return the file as a string you can potentially run  
>> out
>>    of memory if the file is too big.
>>
>> I would like to be able to return an iterator that can read from the
>> ZODB. That would probably benefit the WSGI integration as well.  
>> Anyone
>> has ideas about how to solve this?
>>
>> -- 
>> Sidnei da Silva
>> Enfold Systems                http://enfoldsystems.com
>> Fax +1 832 201 8856     Office +1 713 942 2377 Ext 214
>> _______________________________________________
>> Zope-Dev maillist  -  Zope-Dev at zope.org
>> http://mail.zope.org/mailman/listinfo/zope-dev
>> **  No cross posts or HTML encoding!  **
>> (Related lists -
>>  http://mail.zope.org/mailman/listinfo/zope-announce
>>  http://mail.zope.org/mailman/listinfo/zope )
>>
>