-----Original Message----- From: Dieter Maurer [mailto:dieter@handshake.de] Sent: Monday, January 03, 2005 12:29 PM To: Andy Yates Cc: zope@zope.org Subject: RE: [Zope] Zope 2.7.3 Memory Leaks
Andy Yates wrote at 2005-1-3 09:06 -0600:
... I've been watching the debug page in the control panel and the refcounts fluctuate up and down by small amounts but the trend is always up. They never seem to level off, they always climb.
Is this typical or does this suggest a problem?
Flush the ZODB caches ("Control_Panel --> Database Management --> <database> --> "Flush cache" --> "minimize").
After that, all reference counts should be small. If not, you have a leak. The classes with high reference counts probably will give you glues which objects are leaking...
After flushing main and temporary Class January 2, 2005 5:57 pm January 3, 2005 12:54 pm Delta DocumentTemplate.DT_In.InClass 37 53 +16 compiler.misc.Set 10 19 +9 compiler.pyassem.PyFlowGraph 9 15 +6 compiler.visitor.ASTVisitor 8 14 +6 RestrictedPython.RCompile.RestrictedExpressionCodeGenerator 9 14 +5 compiler.ast.Compare 6 11 +5 compiler.pyassem.Block 5 9 +4 pyrad.dictionary.Dictionary 4 8 +4 pyrad.client.Client 4 8 +4 compiler.ast.Const 10 13 +3 compiler.ast.Expression 8 11 +3 DocumentTemplate.DT_With.With 8 11 +3 compiler.symbols.SymbolVisitor 3 6 +3 compiler.ast.Name 18 20 +2 App.ApplicationManager.DebugManager 7 9 +2 compiler.pycodegen.LocalNameFinder 5 7 +2 compiler.misc.Stack 5 7 +2 compiler.pycodegen.ClassCodeGenerator 4 6 +2 DocumentTemplate.DT_Var.Var 25 26 +1 compiler.pycodegen.FunctionCodeGenerator 7 8 +1 compiler.symbols.ModuleScope 4 5 +1 compiler.pyassem.LineAddrTable 4 5 +1 RestrictedPython.RCompile.RestrictedFunctionCodeGenerator 3 4 +1 OK so the numbers are small. This means there is no leak right? It may not be leaking, but python still consumes all available memory and dies unless I restart Zope every few days. ;-)
... Products.Transience.TransientObject.TransientObject 4041 14891 +10850
It is strange that you have that many session objects...
What is your session limit? What is your session exiration time?
Maximum number of subobjects is infinite (0) Data object timeout is 45 minutes Andy
Andy Yates wrote at 2005-1-3 13:49 -0600:
... After flushing main and temporary ... OK so the numbers are small. This means there is no leak right?
Right -- at least not the usually leaks. The reference count list shows you only reference counts for instances (and not elementary types). You might build leaking cycles consisting only of elementary types -- but this is quite unlikely. Also, Python never releases integer objects it has constructed. Thus, once you iterated from 0 to 4 G, you have several GB of integer objects -- but it is unlikely that you do this...
It may not be leaking, but python still consumes all available memory and dies unless I restart Zope every few days. ;-)
You did this flushing and looking at the reference counts in a process that already had run for several days (and therefore was quite huge)? Do you use C extensions that create non-Python objects (and therefore do not have reference counts and show up in the list)? Maybe, such a C extension is leaking. There are tools to analyse memory leaks (and other memory related problems). Once, I have use the (commercial and expensive!) "purify". Andreas (Jung) told me that there are free open source tools for this purpose around. Unfortunately, I forgot the details (I only create standard leaks :-)
... Maximum number of subobjects is infinite (0) Data object timeout is 45 minutes
Are your sessions huge? Usually, they are stored as pickles in RAM. If they were huge, some thousands of sessions might consume a considerable amount of RAM. You would not see them in the reference counts because Pickles are strings and you see only instances. -- Dieter
[Dieter Maurer]
... Also, Python never releases integer objects it has constructed. Thus, once you iterated from 0 to 4 G, you have several GB of integer objects -- but it is unlikely that you do this...
But Python does reuse space allocated for integers when it can. So, for example, this causes no visible memory growth: i = 0 while i < 4*1024**3: i += 1 If it were possible (it isn't) to get away with doing range(4*1024**3) then the space for those 4 billion *simultaneously-alive* integers would indeed stick around forever. Same story for floats, BTW. It's the maximum number simultaneously alive that matters here, not the total number of ints or floats ever created. In any case, I sure agree this is unlikely to be "the problem" that's biting Andy.
participants (3)
-
Andy Yates -
Dieter Maurer -
Tim Peters