[Zope] Alpha of ODP Viewer product

[Amos Latteier]

At 08:09 PM 8/15/99 -0400, Andrew M. Kuchling wrote:
>Maik Roeder writes:
>>What do you mean by "get the *real* XML dumps into Zope" ?
>
>Various pieces don't expect to handle 45Mb of input data.  For
>example, ZPublisher.Client handles file uploads by reading all the
>data into memory, and it dies; pcgi-wrapper tries to read all of stdin
>and also dies, etc.  I'm trying to do it from the monitor_client with
>the following incantation:
>
>import Zope ; a=Zope.app()
>a.manage_addProduct['XMLDocument'].add(a,
>                   id='s1', title='Structure',
>                   file=open('structure.rdf', 'r') )
>
>I haven't made this work yet, and suspect this is also choking
>somewhere along the way.  Anyone from Digicool ever tried to shovel
>this much data into an XMLDocument?

I know that this is a problem area. Unfortunately XML Document is an
unsupported products, and I only work on it during "spare" time.
Nevertheless, I spent a little time on it just now, and have checked in
some changes that seem to help. At least it takes a lot longer to run out
of memory now ;-)

Now XMLDocuments use temporary commits to help keep memory from getting out
of hand while parsing large uploads. Also I fixed a type marshalling
problem that should keep large uploaded files as temp files on disk, rather
than loading them into memory.

There still seem to be some issues in both ZServer's handling of huge
uploads and XML Document's parsing of huge files which could be improved...

I don't quite understand the ZODB tricks that the File object uses to make
sure you don't run out of memory. When I can get Jim to explain to me
what's going on, I'll try to make XML Document work better with huge files.

-Amos

P.S. I'm very excited about the ODP viewer product! I wonder if it could be
incorporated into the new Zope site...