[ZODB-Dev] Cache Query (why doesn't RAM usage ever drop?)

[Chris Withers]

Toby Dickenson wrote:
> On Friday 25 October 2002 1:12 pm, Chris Withers wrote:
> 
>>apart from the online help now being out of
>>date ;-)
> 
> I thought I had got most of them. Let me know which, and ill fix it.

I just hit the help button:
http://x/HelpSys?help_url=/Control_Panel/Products/OFSP/Help/Database-Management_Database.stx

>>Now, I tried flushing the cache,
> 
> by pressing the 'Minimize' button?

Yup.

> 'target' may not be the best measure. can you post the whole data area of your 
> 'cache parameters' page

Total number of objects in the database           17384
Total number of objects in memory from all caches 1340
Target number of objects in memory per cache      400

Total number of objects in each cache:
Cache Name               Number of object in memory Number of ghost objects
<Connection at 089d227c> 377                        725
<Connection at 089f4324> 312                        471
<Connection at 0c5129fc> 321                        471
<Connection at 0ed99b6c> 330                        482

Total
1340

> That can happen if you have high memory pressure. For a definition of 'memory 
> pressure' see http://www.zope.org/Members/htrd/cache/why .

Fascinating :-)

Sadly, I see the most crucial problem hasn't really been addressed: you can 
still top a server by having, say, a random python script that drags in a stupid 
number of objects between transaction boundaries. I wonder if it'd be possible 
to set a hard limit on the memory used by Zope and somehow get the cache to 
enforce it?

> first, target size is only enforced at garbage collection time; in Zope this 
> is at the end of each transaction or subtransaction. Between these point, the 
> cache can grow arbitrarily large.

...that's my point ;-)

> No way. If you are doing some ZCatalog reindexing then you are mostly dealing 
> with objects sized a few hundred bytes.

So if I've got a 0.5Mb text file, it doesn't have to be loaded into memory to be 
indexed?

> Do you really have any objects that create >500k ZODB objects? If so, thats a 
> design flaw in that class.

Nah, it's a PDF that needs indexing ;-)

>>and in there anything that can be doen to make that happen?
> 
> Restart Zope?

That's the current thinking, but that's a but scummy...

cheers,

Chris