RE: [Zope] Zope 2.7.3 Memory Leaks
Still trying to track down an apparent memory leak that started after upgrading to 2.7.3.
-----Original Message----- From: zope-bounces@zope.org [mailto:zope-bounces@zope.org] On Behalf Of Andy Yates Sent: Thursday, December 02, 2004 4:24 PM To: zope@zope.org Subject: [Zope] Zope 2.7.3 Memory Leaks
We recently upgraded from: Zope Version (Zope 2.6.1 (source release, python 2.1, linux2), python 2.2.2, linux2) Python Version 2.2.2 (#1, Feb 24 2003, 19:13:11) [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)]
To: Zope Version (Zope 2.7.3-0, python 2.3.4, linux2) Python Version 2.3.4 (#1, Sep 14 2004, 10:22:31) [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)]
Now it seems to leak memory (400+ Mb) to the point I have to restart Zope every 3 days.
If Zope is not restarted python2.3 will consume all physical memory and then start using swap. The server slows to a crawl and Zope will eventually stop serving pages. I've been watching the debug page in the control panel and the refcounts fluctuate up and down by small amounts but the trend is always up. They never seem to level off, they always climb. Is this typical or does this suggest a problem? I wrote a new test script in /usr/lib/Zope-2.7/lib/python/Products/Transience/tests that creates sessions with large arrays of random data. Running this test did not seem to cause any leaks. However, if I uses sessions in Zope, Zope seems to leak. The larger my session data the faster it leaks. Here is the output from the debug control panel. # Zope version: (Zope 2.7.3-0, python 2.3.4, linux2) # Python version: 2.3.4 (#1, Sep 14 2004, 10:22:31) [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] # System Platform: linux2 # SOFTWARE_HOME: /usr/lib/Zope-2.7/lib/python # INSTANCE_HOME: /var/zope # CLIENT_HOME: /var/zope/var # Process ID: 22575 (1326767408) # Running for: 16 hours 36 min 56 sec # sys.path: /var/zope/lib/python /usr/lib/Zope-2.7/lib/python/Zope/Startup /usr/lib/Zope-2.7/lib/python /usr/lib/python23.zip /usr/lib/python2.3 /usr/lib/python2.3/plat-linux2 /usr/lib/python2.3/lib-tk /usr/lib/python2.3/lib-dynload /usr/lib/python2.3/site-packages Class January 2, 2005 5:57 pm January 3, 2005 8:36 am Delta Products.Transience.TransientObject.TransientObject 4041 14891 +10850 BTrees.OOBTree.OOBTree 2019 11997 +9978 BTrees.OOBTree.OOBucket 1297 5512 +4215 Products.PageTemplates.Expressions.SubPathExpr 6669 9911 +3242 Products.PageTemplates.Expressions.PathExpr 2644 3858 +1214 Products.PageTemplates.ZopePageTemplate.ZopePageTemplate 1542 2027 +485 Shared.DC.Scripts.Bindings.NameAssignments 880 1333 +453 Products.PageTemplates.ZRPythonExpr.PythonExpr 1062 1411 +349 Products.PythonScripts.PythonScript.PythonScript 679 833 +154 Products.PageTemplates.Expressions.StringExpr 294 448 +154 OFS.Image.Image 488 641 +153 BTrees.IOBTree.IOBucket 74 211 +137 Shared.DC.Scripts.Signature.FuncCode 375 490 +115 RestrictedPython.PrintCollector.PrintCollector 188 303 +115 Products.ZSQLMethods.SQL.SQL 519 629 +110 Shared.DC.ZRDB.DA.SQL 117 198 +81 Shared.DC.ZRDB.Aqueduct.Args 117 198 +81 App.Extensions.NoBrains 114 194 +80 Shared.DC.ZRDB.sqlvar.SQLVar 141 217 +76 DocumentTemplate.DT_HTML.HTML 111 177 +66 OFS.Folder.Folder 547 604 +57 Products.PageTemplates.Expressions.NotExpr 200 254 +54 Persistence.PersistentMapping 106 156 +50 Shared.DC.ZRDB.sqltest.SQLTest 85 134 +49 OFS.Image.File 32 72 +40 ZPublisher.HTTPRequest.record 15 43 +28 OFS.Image.Pdata 18 44 +26 Products.XMLRPCMethod.XMLRPCMethod.XMLRPCMethod 13 37 +24 Shared.DC.ZRDB.sqlgroup.SQLGroup 35 50 +15 DocumentTemplate.DT_Util.Eval 341 353 +12 OFS.DTMLDocument.DTMLDocument 55 67 +12 Products.PlacelessTranslationService.GettextMessageCatalog.GettextMessag eCatalog 25 35 +10 App.Extensions.FuncCode 24 34 +10 AccessControl.User.User 11 20 +9 App.ApplicationManager.ApplicationManager 22 30 +8 Products.ZMySQLDA.DA.Connection 20 28 +8 Products.SiteErrorLog.SiteErrorLog.SiteErrorLog 32 37 +5 AccessControl.User.UserFolder 31 35 +4 DocumentTemplate.DT_Var.Comment 10 14 +4 Products.MailHost.MailHost.MailHost 8 12 +4 App.Product.ProductFolder 10 13 +3 Products.PlacelessTranslationService.PlacelessTranslationService.Placele ssTranslationService 9 12 +3 App.ApplicationManager.DebugManager 7 9 +2 pyrad.dictionary.Dictionary 4 6 +2 pyrad.client.Client 4 6 +2 DateTime.DateTime.DateTime 71 72 +1 ZServer.medusa.counter.counter 31 32 +1 Products.ZMySQLDA.db.DB 19 20 +1 ZServer.HTTPServer.zhttp_channel 6 7 +1 asynchat.fifo 5 6 +1 Thanks! Andy
Probably a pretty simple question. The following gives me an in my output which is probably what I am telling it to do. I want a simple so that my empty td's have contents when there is nothing in them and my tables will behave in IE. <td tal:define="option_description option/description"> <span tal:condition="python:option_description == ''" tal:content="string: " tal:omit-tag=""> </span> <span tal:condition="python:option_description != ''" tal:content="option_description" tal:omit-tag=""> Description goes here. </span> </td> Can someone advise me what I should be substituting in the string to provide the desired output. Also, is there a better way of handling empty table cells than a separate set of condition statements for each td that may be empty? Regards, David
On Monday 03 January 2005 08:43 am, David Pratt wrote:
Probably a pretty simple question. The following gives me an in my output which is probably what I am telling it to do. I want a simple so that my empty td's have contents when there is nothing in them and my tables will behave in IE.
<td tal:define="option_description option/description"> <span tal:condition="python:option_description == ''" tal:content="string: " tal:omit-tag=""> </span> <span tal:condition="python:option_description != ''" tal:content="option_description" tal:omit-tag=""> Description goes here. </span> </td>
Can someone advise me what I should be substituting in the string to provide the desired output. Also, is there a better way of handling empty table cells than a separate set of condition statements for each td that may be empty?
You can just change 'string: ' to 'structure string: ', so that it is not html quoted. I would also use 'tal:replace' in place of 'tal:content' and 'tal:omit-tag', and perhaps simplify the conditions to 'tal:condition="option_description"' and 'tal:condition="not: option_description"' (unless you explicitly want to show non-string false values like 0, False, [], {}, (), None as strings, which you may). Alec Mitchell
On Mon, Jan 03, 2005 at 09:11:07AM -0800, Alec Mitchell wrote:
On Monday 03 January 2005 08:43 am, David Pratt wrote:
Probably a pretty simple question. The following gives me an in my output which is probably what I am telling it to do. I want a simple so that my empty td's have contents when there is nothing in them and my tables will behave in IE.
<td tal:define="option_description option/description"> <span tal:condition="python:option_description == ''" tal:content="string: " tal:omit-tag=""> </span> <span tal:condition="python:option_description != ''" tal:content="option_description" tal:omit-tag=""> Description goes here. </span> </td>
Can someone advise me what I should be substituting in the string to provide the desired output. Also, is there a better way of handling empty table cells than a separate set of condition statements for each td that may be empty?
You can just change 'string: ' to 'structure string: ', so that it is not html quoted. I would also use 'tal:replace' in place of 'tal:content' and 'tal:omit-tag' ... (snip)
WHy not leave out tal:replace or tal:content entirely? <span tal:condition="python:option_description == ''" tal:omit-tag=""> </span> That has the effect I think you're after. -PW -- Paul Winkler http://www.slinkp.com
The solutions are good. Many thanks for ZPT help. Regards, David
David Pratt wrote at 2005-1-3 12:43 -0400:
Probably a pretty simple question. The following gives me an in my output which is probably what I am telling it to do. I want a simple so that my empty td's have contents when there is nothing in them and my tables will behave in IE.
Indeed. Use the "structure" keyword to tell "tal:content/replace" not to perform HTML quoting... Read the Zope Book (2.6/2.7 edition, online) in case you are unfamiliar with "structure"... -- Dieter
Andy Yates wrote at 2005-1-3 09:06 -0600:
... I've been watching the debug page in the control panel and the refcounts fluctuate up and down by small amounts but the trend is always up. They never seem to level off, they always climb.
Is this typical or does this suggest a problem?
Flush the ZODB caches ("Control_Panel --> Database Management --> <database> --> "Flush cache" --> "minimize"). After that, all reference counts should be small. If not, you have a leak. The classes with high reference counts probably will give you glues which objects are leaking...
... Products.Transience.TransientObject.TransientObject 4041 14891 +10850
It is strange that you have that many session objects... What is your session limit? What is your session exiration time? -- Dieter
participants (5)
-
Alec Mitchell -
Andy Yates -
David Pratt -
Dieter Maurer -
Paul Winkler