understanding Zope's memory usage
I am running a server with a handfull of plone intranet sites. The server has 4GB memory, but is now starting to page. What I notice is that the different Plone instances vary widely in there memory usage. For one site every process uses 400Mb, and there are 6 processes running! That is 2.4 Gbyte for one site! I see there is one deamon process that is small, and 6 processes that are big. If we call the six biggies a-f: deamon start a a starts b b starts c d e f c d e f are probably zserver threads, but what do process a and b? And why do the all have the same size and RSS? What are the alternatives to limit the memory consumption? Anybody who can enlighten me on this? Thanks, Lucas
--On Mittwoch, 13. April 2005 10:51 Uhr +0200 Lucas Hofman <lucas.hofman@pgs.com> wrote:
I am running a server with a handfull of plone intranet sites. The server has 4GB memory, but is now starting to page.
What I notice is that the different Plone instances vary widely in there memory usage. For one site every process uses 400Mb, and there are 6 processes running! That is 2.4 Gbyte for one site!
Zope runs only in one process with multiple threads (where threads on some OSes are implemented as processes :-))
I see there is one deamon process that is small, and 6 processes that are big. If we call the six biggies a-f: deamon start a a starts b b starts c d e f
c d e f are probably zserver threads, but what do process a and b? And why do the all have the same size and RSS? What are the alternatives to limit the memory consumption?
a or b is likely the zdaemon spawning up the threads.
What are the alternatives to limit the memory consumption?
Restarting Zope regularly (at night) might be a wise choice (this is a *serious* advice). -aj
Andreas Jung wrote:
--On Mittwoch, 13. April 2005 10:51 Uhr +0200 Lucas Hofman <lucas.hofman@pgs.com> wrote:
I am running a server with a handfull of plone intranet sites. The server has 4GB memory, but is now starting to page.
What I notice is that the different Plone instances vary widely in there memory usage. For one site every process uses 400Mb, and there are 6 processes running! That is 2.4 Gbyte for one site!
Zope runs only in one process with multiple threads (where threads on some OSes are implemented as processes :-))
I was tricked by the output of ps and top to believe that all these were processes. In addition, looking at the memory usage with 'free' gave me more insight in what linux is actually doing. Thanks.
Restarting Zope regularly (at night) might be a wise choice (this is a *serious* advice).
Will do. A bit of a nuisance with offices and vessels all over the globe accessing the site. Actually the 400GB process was only running for one day. Was there not a Zope version with some serious memory leaks that were fixed? (these sites are running different python/zope/plone versions) Lucas
--On Mittwoch, 13. April 2005 11:33 Uhr +0200 Lucas Hofman <lucas.hofman@pgs.com> wrote:
Will do. A bit of a nuisance with offices and vessels all over the globe accessing the site. Actually the 400GB process was only running for one day. Was there not a Zope version with some serious memory leaks that were fixed? (these sites are running different python/zope/plone versions)
Which Zope version are you running? Since writing leaking code in Zope can be easy this problem can be anywhere and it is often hard to track it down. -aj
Andreas Jung wrote:
--On Mittwoch, 13. April 2005 11:33 Uhr +0200 Lucas Hofman <lucas.hofman@pgs.com> wrote:
Will do. A bit of a nuisance with offices and vessels all over the globe accessing the site. Actually the 400GB process was only running for one day. Was there not a Zope version with some serious memory leaks that were fixed? (these sites are running different python/zope/plone versions)
Which Zope version are you running? Since writing leaking code in Zope can be easy this problem can be anywhere and it is often hard to track it down.
python 2.3.4 - zope 2.7.2 - plone 2.0.4 python 2.4.1 (I know, should have used 2.3.5) - zope 2.7.3 - plone 2.0.5 The sites with most memory consumption are the oldest (but then, they have more content in their Data.fs) Lucas
--On Mittwoch, 13. April 2005 11:45 Uhr +0200 Lucas Hofman <lucas.hofman@pgs.com> wrote:
Andreas Jung wrote:
--On Mittwoch, 13. April 2005 11:33 Uhr +0200 Lucas Hofman <lucas.hofman@pgs.com> wrote:
Will do. A bit of a nuisance with offices and vessels all over the globe accessing the site. Actually the 400GB process was only running for one day. Was there not a Zope version with some serious memory leaks that were fixed? (these sites are running different python/zope/plone versions)
Which Zope version are you running? Since writing leaking code in Zope can be easy this problem can be anywhere and it is often hard to track it down.
python 2.3.4 - zope 2.7.2 - plone 2.0.4
You should upgrade to 2.7.5. Older versions have a severe memory leak in cAccessControl (see release notes).
python 2.4.1 (I know, should have used 2.3.5) - zope 2.7.3 - plone 2.0.5
same here -aj
Andreas Jung wrote:
--On Mittwoch, 13. April 2005 11:45 Uhr +0200 Lucas Hofman <lucas.hofman@pgs.com> wrote:
Andreas Jung wrote:
--On Mittwoch, 13. April 2005 11:33 Uhr +0200 Lucas Hofman <lucas.hofman@pgs.com> wrote:
Will do. A bit of a nuisance with offices and vessels all over the globe accessing the site. Actually the 400GB process was only running for one day. Was there not a Zope version with some serious memory leaks that were fixed? (these sites are running different python/zope/plone versions)
Which Zope version are you running? Since writing leaking code in Zope can be easy this problem can be anywhere and it is often hard to track it down.
python 2.3.4 - zope 2.7.2 - plone 2.0.4
You should upgrade to 2.7.5. Older versions have a severe memory leak in cAccessControl (see release notes).
Andreas, can I use the same versions of Plone and all other products when upgrading zope to 2.7.5? Will the Data.fs load without problems? Lucas
--On Mittwoch, 13. April 2005 13:26 Uhr +0200 Lucas Hofman <lucas.hofman@pgs.com> wrote:
Andreas, can I use the same versions of Plone and all other products when upgrading zope to 2.7.5? Will the Data.fs load without problems?
Backup first, try it out :-) In general it should work. -aj
participants (2)
-
Andreas Jung -
Lucas Hofman