[Zope] Memory Leak Debugging

[Peter Bengtsson]

Hi,
I'm NOT an expert and I hope that someone else who is an expert gives
you better advise but here's a start...
Do you have any recursive loops with lists that doesn't reset the
list? Perhaps something like this:

def findAllImages(self, placetolook, append2):
    for o in placetolook.objectValues():
        if o.meta_type == 'Silva Image':
            append2.append(o)
        elif o.isPrincipiaFolderish:
            findAllImages(o, append2)
or something like that.

I had a memory leak and the refcounts pointed to the Photo class.
Turns out that I did a bizarre loop over these without resetting it
properly after. It was a bitch to debug and it was a long time ago
when Zope was called zope 2.3.

That's all I've got to offer. Scan your code for places where you
create lists by loops on the ZODB. 2000 objects is enough if they are
piled up and not freed until a server restart. Heck, if they're big,
200 is enough.

On 2/15/06, Jonathan Bowlas <me at jonbowlas.com> wrote:
> Hi All,
>
> We're having serious problems with performance running Silva CMS on Zope
> 2.7.8-final, python 2.3.5, linux2 set up on a machine with 2.5GB RAM. There
> are currently only about 65 users and around 2000 documents in the CMS but
> it seems to run for a week, then the RAM fills up and crashes so we have to
> restart the server manually.
>
> The Zope logs aren't much help as I enquired on the Zope list and their
> response was we need more RAM. But surely 2.5GB is enough.
>
> I've also checked Control_Panel/DebugInfo for any refcounts with significant
> jumps and Products.Silva.Image.Image seems to be the culprit:
>                                        11.32am  11.36am
> Products.Silva.Image.Image                 56203        58153      +1950
> Acquisition.ImplicitAcquirerWrapper        44825        45151      +326
>
> So we've tried to figure out which kind of request triggers these refcount
> jumps by creating a separate instance with a very small ZODB Cache (so it
> gets filled up soon), and clicked around a little, watching the refcounts
> but we cannot seem to replicate it.
>
> So basically this is a plea for help from an expert Zope debugger who can
> assist us in finding the source of the problem so we can resolve it. Or
> alternatively someone who can give us a set of instructions that we can
> follow that will enable us to locate the problem, I've googled it and there
> are no clear answers.
>
> FYI we have the following products installed on our Zope instance.
>
> Annotations 0.4.3
> BTreeFolder2-1.0.1
> DocFinderTab (0.5.2)
> External Method-1-0-0
> FileSystemSite 1.4.2
> Formulator 1.10
> Groups 0.4
> IssueTrackerProduct (0.6.13)
> LDAPUserFolder (2.3)
> MIMETools
> MailHost-1-3-0
> OFSP-1-0-0
> PageTemplates-1-4-0
> ParsedXML 1.4
> PlacelessTranslationService (1.0)
> PluginIndexes
> ProxyIndex 1.2
> PythonScripts-2-0-0
> Sessions
> Silva 1.4.1
> SilvaBlog 0.4.2
> SilvaCustomAutoTOC 0.9
> SilvaDocument 1.4.1
> SilvaDocumentPatches 1.3b6-1
> SilvaExtETHLDAP 0.7
> SilvaExternalSources 0.10.3
> SilvaMetadata 0.10
> SilvaNewsNetwork 2.0
> SilvaReferenceChecker 0.2
> SilvaStylesheets 0.6.2
> SilvaUCLAttributes 0.7 - (A UCL product*)
> Silva UCL Gallery 0.1 - (A UCL product*)
> SilvaUCLSubjectAudienceFilters 0.6 - (A UCL product*)
> SilvaViews 0.11
> SiteAccess-2-0-0
> SiteErrorLog
> Sprout 0.6.2
> StandardCacheManagers-1-1-0
> TemporaryFolder
> Transience
> Tropos (1.9.12)
> XMLWidgets-Pluggable 0.13 (CVS/UNRELEASED)
> ZCTextIndex
> ZCatalog-2-2-0
> ZGadflyDA
> ZMySQLDA 2.0.8
> ZODBMountPoint
> ZOracleDA
> ZReST (1.1)
> ZSQLMethods
> Zope Tutorial 1.2
> kupu 1.3.2
> z3locales 0.1
>
> If you require any further information do not hesitate to ask.
>
> Hope someone can help.
>
> Jon Bowlas
> University College London
>
> _______________________________________________
> Zope maillist  -  Zope at zope.org
> http://mail.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists -
>  http://mail.zope.org/mailman/listinfo/zope-announce
>  http://mail.zope.org/mailman/listinfo/zope-dev )
>


--
Peter Bengtsson,
work www.fry-it.com
home www.peterbe.com
hobby www.issuetrackerproduct.com