Florent Guillaume wrote:

The RAMCacheManager does a costly pseudo-pickling of the objects it stores to compute their size, but that information is only used in the statistics screen.

The motivation was actually more subtle: I wanted to prevent applications from caching things that weren't self-contained. As I saw it, I would have otherwise spent countless hours debugging cache problems, and the caching framework would have ultimately been rejected as too fragile--similar to the fate of the Refresh product. Catalog results in particular are an obvious thing to cache, but they aren't safe for caching because they link back to the catalog. You'd have major thread problems and probably inconsistent results. The pickle machinery throws an exception when it hits many kinds of objects that aren't self-contained, so it made a good match. The only waste is the time spent building a string. It was only lucky that the string could also be used for estimating memory consumption. I think RAMCacheManager should, by default, continue to make a reasonable attempt to ensure objects are safe for caching, but it might be good to also have an advanced option that disables the filter.

2. or, how about using the size to have a cache threshold based on the size. That would help manage memory consumption better. Has anybody done something like that?

I don't think that has been attempted, but I agree it would be interesting. Shane

Florent Guillaume wrote at 2005-7-8 20:36 +0200:

The RAMCacheManager does a costly pseudo-pickling of the objects it stores to compute their size, but that information is only used in the statistics screen.

I replaced it by the following code: try: from cPickle import Pickler, HIGHEST_PROTOCOL except: from pickle import Pickler, HIGHEST_PROTOCOL ... class CacheEntry: ''' Represents a cached value. ''' def __init__(self, index, data, view_name): try: # This is a protective barrier that hopefully prevents # us from caching something that might result in memory # leaks. It's also convenient for determining the # approximate memory usage of the cache entry. # DM 2004-11-29: this code causes excessive time. # Note also that it does not prevent us from # caching objects with references to persistent objects # When we do, nasty persistency errors are likely # to occur ("shouldn't load data while connection is closed"). #self.size = len(dumps(index)) + len(dumps(data)) sizer = _ByteCounter() pickler = Pickler(sizer, HIGHEST_PROTOCOL) pickler.dump(index) pickler.dump(data) self.size = sizer.getCount() .... class _ByteCounter: '''auxiliary file like class which just counts the bytes written.''' _count = 0 def write(self, bytes): self._count += len(bytes) def getCount(self): return self._count -- Dieter