[Zope] Controlling caching of objects

[Toby Dickenson]

(I trimmed the crossposts)

On Wednesday 22 October 2003 09:05, Bjorn Stabell wrote:
> In Zope 2.6.2 it's quite easy to put an upper limit on the number of
> objects in memory per thread.  But does this mean that Zope won't swap
> out objects until it reaches that limit?

The right term is "deactivate". The main reason for deactivating an object is 
that the total number of objects exceeds the per-thread limit. There are 
other reasons.

> It looks like this:
>
>     Target # of objects per thread:    20,000
>     Actual # of objects per thread:    ~15,000
>     Actual # of ghost objects per thread:    ~43,000
>
> Were the ghost objects once loaded in memory and then swapped out? 

No. Ghost objects are those that need a small (100 byte approx) stub in memory 
because there is an active object that has a reference to it. The difference 
between a ghost (stub) and a full object is that the ghost object does not 
have all its attributes loaded into memory. It does has its oid in memory 
though, so it can ask the storage to load those attributes on demand. This is 
called "activation".

Note that if one of those attributes is a reference to another persistent 
object (for example, think of folders), then activation will trigger the 
creation of a ghost for each referenced object.

Ghosts are removed from memory based on reference counting, and is the inverse 
of the process described in the previous paragraph.

> I find the current system of controlling # of objects in memory per
> thread to be a bit difficult to use.  Most people, I think, would prefer
> to put an upper limit on the total MB of memory Zope uses instead.  Are,
> what are the cons/pros?

pro: it would be much better.
con: it hasnt been done yet.

The current system is much better than in versions before 2.6. Noone has found 
the current system bad enough to give it the effort needed for improvement. 
Patches gratefully accepted.

-- 
Toby Dickenson