[Zope] [ZODB-Dev] Zope memory usage

[Manuel Vazquez Acosta]

Alan,

I'm replying to the Zope list also, because this issue is perhaps
related to other components there.

I'm running into the same situation: The python process running my Plone
site is steadyly growing.

I'm using Zope2.9.8-final (the one which works with Plone 2.5.5).

What is the plan for Zope include such feature?

Best regards,
Manuel.

Alan Runyan wrote:
> There was a recent modification to limit the ZODB cache to a set size.  i.e.
> Limit the size of memory usage to 128MB.
> 
> The original feature was implemented here:
>   http://svn.zope.org/ZODB/branches/dm-memory_size_limited-cache/
> 
> You can get the feature+3.8 branch of the ZODB from:
>   http://svn.zope.org/ZODB/branches/zcZODB-3.8/
> 
> The changes are also on trunk (will be ZODB 3.9).
> 
> The goal of the modification is to try to put upper bounds on the size of
> the database cache.  Before this "size limited cache" the cache would grow
> and if you had very large objects in the zodb cache -- your memory usage would
> be surpringsly high.
> 
> Please use this feature in your testing at upfront.  And let Roche know this
> feature has landed (if we was not monitoring svn checkins).  The feature needs
> to be test driven.  Give it a go and report your experience.
> 
> cheers
> alan
> 
> 
> On Wed, Sep 17, 2008 at 5:10 AM, Izak Burger <izak at upfrontsystems.co.za> wrote:
>> Hi all,
>>
>> I'm sure this question has been asked before, but it drives me nuts so I
>> figured I'll ask again. This is a problem that has been bugging me for
>> ages. Why does zope memory use never decrease? Okay, I've seen it
>> decrease maybe by a couple megabyte, but never by much. It seems the
>> general way to run zope is to put in some kind of monitoring, and
>> restart it when memory goes out of bounds. In general it always uses
>> more and more RAM until the host starts paging to disk. This sort of
>> baby-sitting just seems wrong to me.
>>
>> It doesn't seem to make any difference if you set the cache-size to a
>> smaller number of objects or use a different number of threads. Over
>> time things always go from good to bad and then on to worse. I have only
>> two theories: a memory leak, or an issue with garbage collection (python
>> side).
>>
>> It is possible that this is not a bug, my reasoning goes like this: The
>> OS exposes a "virtual memory" picture to the application, ie, the
>> application does not know how much of the RAM is real and how much is
>> paged out. All it does is call malloc every now and then. Combined with
>> a kernel that allows overcommit (which is in general a good thing), I
>> see this going only one way.
>>
>> I'm hoping someone on this list has some insights into the issue.
>>
>> regards,
>> Izak
>> _______________________________________________
>> For more information about ZODB, see the ZODB Wiki:
>> http://www.zope.org/Wikis/ZODB/
>>
>> ZODB-Dev mailing list  -  ZODB-Dev at zope.org
>> http://mail.zope.org/mailman/listinfo/zodb-dev
>>
> 
> 
>