[ZODB-Dev] ZODB and ORBit-Python problems

[Jim Fulton]

Greg Ward wrote:
> 
> On 18 May 2001, Christian Robottom Reis said:
> > How fast is it, Greg? The ZODB/ZEO combo, I mean, regardless of my
> > problem? I'm running some tests and reading all objects seems pretty slow.
> 
> It's fast enough.  Our database is currently ~30 MB, ~100k objects.  We
> generally access it through ZEO with the client and server on the same
> machine, communicating over a TCP socket.  There seem to be two ways to
> traverse an entire ZODB:
>   * start at the root and follow all pointers (list/dict elements,
>     instance attributes) -- ie. object-graph traversal
>   * start at OID 0 ('\0\0\0\0\0\0\0\0') and fetch every object
>     from the database with dictionary-style lookups on the
>     Connection
> 
> On my development machine (800 MHz Athlon, 256 MB RAM, Linux 2.2.19,
> Python 2.1, and whatever version of ZODB/ZEO Andrew keeps us at --
> presumably the latest from zodb.sourcefourge.net, but I dunno), it
> takes:
>   * ~7 minutes to walk the object graph, type-checking every last
>     scrap of data that we hit (wow: I just noticed that the type-
>     checking process grows to ~80 MB of memory -- good thing I
>     splurged for 256 MB in this machine!)

You could modify your script to your script to "ghostify"
the objects, by calling _p_deactivate on them after you use them
to prevent the memory bloat.  This is what the Zope "find" facility
does when searching for objects.

Of course, if you want to load all your data in memory at once, you don't need
ZODB.  Where ZODB tries to add value is by loading and keeping just the data 
you need when you need it.

Jim

--
Jim Fulton           mailto:jim@digicool.com   Python Powered!        
Technical Director   (888) 344-4332            http://www.python.org  
Digital Creations    http://www.digicool.com   http://www.zope.org