[Zope-dev] Cache code in Shared/DC/ZRDB/DA.py

[Dieter Maurer]

Chris Withers wrote at 2006-11-16 17:29 +0000:
>....
>I'm wondering if you can still remember the rational behind the cache 
>code at around lines 355-387 of:
>
>http://svn.zope.org/Zope/trunk/lib/python/Shared/DC/ZRDB/DA.py?rev=68158&view=auto
>
>This code is pretty old (checked in 5th Dec 1997) but has started 
>causing a few people problems under high load:
>
>http://mail.zope.org/pipermail/zope-db/2006-September/004684.html
>http://www.zope.org/Collectors/Zope/2212
>
>In particular:
>
>- in line 368, why is len(cache)>max_cache/2 used as a trigger to start 
>cache clearing? (the /2 in particular)

You may want to start getting rid of keys old enough that you would not
use them anyway already before you reached max cache.

I do not know why that is not done always -- an
"XYBucket" has an efficient method to determine the minimal key,
thus we could get rid of the "len(cache) > max_cache /2".

And of course, it is quite stupid to determine the keys and reverse
them, even when no key is old enough....

>- does it matter that IOBTree.Bucket has gone away and that tcache is
>   now a simple dictionary? It certainly seems to make the keys.reverse()
>   on line 370 superfluous and the keys[-1]<t on line 371 less reliable.

Yes. You want to kill the dict and use the "IOBucket"...

>More generally, do you or does anyone else have any attachment to this 
>code or would anyone mind if I ripped it out and replaced it with 
>something simpler, with more comments and unit tests?

Very good idea.


If you are at it.

The idea to have this cache maintained via a "_v_" attribute
is insane. A module level cache would be much more efficient
and save memory as well.


You may look at my "CCZSQLMethods"...



-- 
Dieter