[Zope] Memory and Large Zope Page Templates

[Chris Withers]

Santi Camps wrote:

>> IIRC, I think that once the size of your process increases, it won't 
>> shrink... however, this does not mean that it will not reuse the same 
>> memory again...
>>
> Dou you know if this is a python behaviour ?  Or a generic one in 
> process managements ?

I'm going to stick my neck out and disagree with Dieter, I remember this 
being an artefact of the python memory manager...

> Yes, that's the situation when the report is executed by the same 
> thread.  But If some reports are executed at the same time in different 
> threads, then the memory size is increased again,  aprox to  30Mb + 80Mb 
> * simultaneous_threads

That sounds odd, although not really. Each thread keeps its own ZODB 
object cache. This cache has a target size which is vaguely tunable, but 
if a singly transaction loads lots of objects (as your report likely 
does) they will all end up being loaded regardless of your cache 
setting. The trick is not to load so many objects into memory (see low 
level ZODB methods for synching, subtransactions, etc) or to make the 
objects more lightweight (see ZCatalogs and brains in particular)

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk