[ZODB-Dev] working with large databases

[Greg Landrum]

At 01:33 AM 2/28/2002, Toby Dickenson wrote:

>How often should you call cacheGC? Thats where my patch comes in.
>Without my patch you need to call it lots, and will pay a large
>performance penalty.
>
>With my patch it doesnt matter too much how often it is called, as
>long as it is often enough. Calling every time round the loop will be
>fine, maybe every 10 times or 100 times will save a few % of
>performance.

I had to apply parts of the patch by hand because the versions of DB.py and 
Connection.py in the diff file do not correspond to any versions I could 
actually find.  The result builds and works under linux.  On windows it 
dumps core as soon as I try and do a commit.  This makes me kind of sad 
because my big memory machine is running Windows.  I'd love to be able to 
trace this down, but getting a full debug build of everything I need is a 
nightmare I don't want to relive right now.

I'm happy to report that, on linux, using Toby's patch and a call to 
_p_jar.cacheGC() every thousand times through the loop reduces memory 
requirements substantially.  The performance penalty is about 15% (that's a 
rough number), which isn't half bad.  Color me pleased.

Now if I could just make it work on windows...

Thanks for the help,
-greg



----
greg Landrum (greglandrum@earthlink.net)
Software Carpenter/Computational Chemist