[Zope] Re: Zope 2.7.3 Memory Leaks

[Tres Seaver]

Andy Yates wrote:
>>You did this flushing and looking at the reference counts in
>>a process that already had run for several days (and therefore was
> 
> quite
> 
>>huge)?
>>
> 
> It took about 24 hours.
>  
> 
>>Do you use C extensions that create non-Python objects (and
>>therefore do not have reference counts and show up in the list)?
>>Maybe, such a C extension is leaking.
> 
> No we do not use C extensions.  In fact we can reproduce this behavior
> on a fresh install.  On a fresh system create a python script that puts
> data in a session object.
> 
> s = context.REQUEST.SESSION
> t = ' ' * 1024
> s['data'] = t
> print "foo"
> return printed
> 
> Then call this script with your favorite bench marking program.  We used
> apache bench (ab).  This will cause python2.3 to consume all available
> memory and crash or lockup.  The speed at which this happens depends on
> the amount of data you assign to 't' and how fast you call the script.
> The session timeout does not seem to affect the memory use.  We even
> tried setting the session timeout to 1 minute.
> 
> In a very basic sense this is all our production server does.  When a
> user first comes to our site we query a mysql database for several
> hundred values.  These values are then stored in the session object as a
> map.  All other pages are built dynamically based on the values stored
> in the session object.  
> 
> Is the session object system just not supposed to be used like this?

We need to do some back-of-the-envelope math here:

   Estimated bytes / entry:      16
   Estimated entries / user:    150
   Estimated overhead / user:   600
   ================================
   Estimated bytes / session:  3000

How many simultaneous sessions are we talking about?  A few hundred 
simultaneous users could chew up RAM pretty fast.  What is your "max 
number of sessions" value?

Tres.
-- 
===============================================================
Tres Seaver                                tseaver at zope.com
Zope Corporation      "Zope Dealers"       http://www.zope.com