[ZODB-Dev] Cache Query (why doesn't RAM usage ever drop?)

[Toby Dickenson]

On Friday 25 October 2002 1:12 pm, Chris Withers wrote:
> Hi,
>
> Just got one of our bigger apps up to 2.6.0 and have been sniffing arou=
nd
> Toby's new cache. Looks cool,=20

wooohoo.

> apart from the online help now being out of
> date ;-)

I thought I had got most of them. Let me know which, and ill fix it.

(one of my gripes about Zope now is that we have got too much documentati=
on.=20
It needs to be distilled)

> Now, I tried flushing the cache,

by pressing the 'Minimize' button?

> but the memory usage didn't drop.

Shane explained why memory usage never drops.

> I'm also concerned that with the target number of objects set to only 4=
00,

'target' may not be the best measure. can you post the whole data area of=
 your=20
'cache parameters' page

> this instance still managed to grow to 225Mb.

That can happen if you have high memory pressure. For a definition of 'me=
mory=20
pressure' see http://www.zope.org/Members/htrd/cache/why . For a graph th=
at=20
relates memory pressure to memory usage, see figure 2 at that same page. =
Did=20
you have subtransactions enabled? If not, then you are effectively have=20
arbitrarily high memory pressure. =20

> Hmm, that said, that would
> mean that at some stage I had 400

No, you misunderstand the meaning of 'target size'.

first, target size is only enforced at garbage collection time; in Zope t=
his=20
is at the end of each transaction or subtransaction. Between these point,=
 the=20
cache can grow arbitrarily large.

second, the target size is per thread. More threads means more memory usa=
ge.

(Thats why the 'use more threads' intuition from people familiar with Apa=
che=20
is bogus. Zope is usually better off with fewer threads, and a bigger cac=
he)

> 0.5Mb objects in memory. On this server,
> that's perfectly believeable...

No way. If you are doing some ZCatalog reindexing then you are mostly dea=
ling=20
with objects sized a few hundred bytes.

Do you really have any objects that create >500k ZODB objects? If so, tha=
ts a=20
design flaw in that class.

> Anyway, does anyone know why memory isn't being returned to the OS by
> python=20

Shane explained why this doesnt happen automatically.=20

> and in there anything that can be doen to make that happen?

Restart Zope?

If this phenomenon only happens after a site-wide recatalog, then I dont=20
consider this an unreasonable remedy.