[Zope-dev] Serving large files

[Christian Theune]

Hi,

Sidnei da Silva wrote:
> I remember having a conversation with Jim at some point where he
> proposed a strategy for requests that could potentially take a long
> time to finish. If I recall correctly, he proposed having a separate
> ZODB connection pool.
> 
> One thing that is problematic today is serving large files
> from the ZODB (ignoring the upcoming blob support).
> 
> a) You can't return an iterator that reads from the ZODB, because by
>    the time the iterator is consumed the connection has already been
>    closed. I believe the iterator is not consumed in the same thread,
>    which can cause yet more issues.
> 
> b) If you dump the data to a temp file and return that as a file
>    iterator, it can potentially take twice the time.
> 
> c) If you use RESPONSE.write() you can break other
>    applications. ExternalEditor comes to mind.
> 
> d) If you just return the file as a string you can potentially run out
>    of memory if the file is too big.
> 
> I would like to be able to return an iterator that can read from the
> ZODB. That would probably benefit the WSGI integration as well. Anyone
> has ideas about how to solve this?

I wonder whether this statement is true:

   Any connection handed out after the connection breaks the ACID
   compliance of the request.

I think this is because:

   When finishing the actual work we have to either commit or abort the
   running transaction.

   In the time until we set up another transaction, there already might
   have been a change to the database by a third transaction, making the
   request neither isolated nor atomic.

My statement is probably only true for all solutions not paying
attention to the statement. ;)

Christian

-- 
gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany
www.gocept.com - ct at gocept.com - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 254 bytes
Desc: OpenPGP digital signature
Url : http://mail.zope.org/pipermail/zope-dev/attachments/20060917/3e298225/signature.bin