[Zope] Memory and Large Zope Page Templates

[Santi Camps]

En/na Chris Withers ha escrit:

> Santi Camps wrote:
>
>>> IIRC, I think that once the size of your process increases, it won't 
>>> shrink... however, this does not mean that it will not reuse the 
>>> same memory again...
>>>
>> Dou you know if this is a python behaviour ?  Or a generic one in 
>> process managements ?
>
>
> I'm going to stick my neck out and disagree with Dieter, I remember 
> this being an artefact of the python memory manager...
>
>> Yes, that's the situation when the report is executed by the same 
>> thread.  But If some reports are executed at the same time in 
>> different threads, then the memory size is increased again,  aprox 
>> to  30Mb + 80Mb * simultaneous_threads
>
>
> That sounds odd, although not really. Each thread keeps its own ZODB 
> object cache. This cache has a target size which is vaguely tunable, 
> but if a singly transaction loads lots of objects (as your report 
> likely does) they will all end up being loaded regardless of your 
> cache setting. The trick is not to load so many objects into memory 
> (see low level ZODB methods for synching, subtransactions, etc) or to 
> make the objects more lightweight (see ZCatalogs and brains in 
> particular)
>
> cheers,
>
> Chris
>
Thanks for your answer.

Yes, I understant how ZODB cache works with threads.   The behaviour I 
didn't understood is why, once transaction is finished, the ZODB Cache 
decrese in number of objects but the RAM process not.

I'm already using brains.  I've tried your suggestion of 
subtransactions, but there is no change (I think it's normal, I'm not 
changing any persistent object).  I'm not sure how a _p_jar.sync() can 
improve something.

I think that the best solution will be run the report in a new process, 
spawning a ZEO Client to do the work and waiting for it.

Regards

Santi Camps