[Zope-dev] Preliminary findings: Zope 2.7 leakage caused by errors

[Andreas Jung]

I have not followed the discussion but is this leak as serious as it should 
be resolved
before the next 2.7.1 beta1 release?

Andreas
--On Donnerstag, 20. Mai 2004 9:52 Uhr +0200 Dieter Maurer 
<dieter at handshake.de> wrote:

> Jean-Francois.Doyon at CCRS.NRCan.gc.ca wrote at 2004-5-19 14:44 -0400:
>> ...
>> Right now I've focused on finding out why the Requests are still around.
>>
>> Using the gc module, I've found that all leaked requests are being held
>> by a dictionary.  If I look at said dictionaries, this is what they
>> might look like:
>>
>> {'REQUEST': <HTTPRequest,
>> URL=/VirtualHostBase/http/atlastest2.ccrs.nrcan.gc.ca:80/VirtualHostRoot
>> /_vh _site/francais/mapsatoz>},
>> {'REQUEST': <HTTPRequest,
>> URL=/VirtualHostBase/http/atlastest2.ccrs.nrcan.gc.ca:80/VirtualHostRoot
>> /_vh _site/francais/mapsatoz>},
>
> These dictionaries look like the "instance dict"s of
> the "ZPublisher.BaseRequest.RequestContainer"s.
>
> When you have any cycle involving an AcquisitionWrapper,
> then its reference to the "RequestContainer" will keep this alive
> together with its instance dict.
>
> You may analyse the content of your requests.
> Do they contain acquisition wrapped objects (or other objects
> with (maybe indirect) references to this dict)?
>
> If not, "REQUEST" does not seem to be the root of your cycles
> but rather a side effect.
>
>
> More important notes:
>
>   *  When you see "AcquisitionWrapper"s in the reference information
>      in "Control_Panel --> Debug Information", then
>      there *must* also be other prominent ExtensionClass'es --
>      the "aq_base" of these wrappers.
>
>      These classes may provide a valuable clue towards the cause
>      of the cycles -- of global variables (by the way).
>
>   *  The reference information in "Control_Panel" covers only
>      instance objects but not elementary Python data types.
>
>      Thus, when cycles are caused by elementary Python objects,
>      we may not directly see them in the list.
>
>      GC's "DEBUG_LEAK" may provide some information.
>
>   *  The leaks may not be caused by cycles at all (though it is
>      the most likely cause) but could result from a global
>      cache containing references to acquisition wrapped objects.
>
>      Such a cache should never exist -- global variables and
>      class attributes must never directly or indirectly reference
>      persistent objects -- as this results in nasty persistency bugs.
>
>      Nevertheless, sometimes software is buggy. I noticed
>      such a cache in Archetypes 1.3a2 (fixed in newer versions).
>
> ....
>> Said dictionary does not seem to be referred to by anything, at least not
>> anything the gc module can find (gc.get_referrers()).
>
> That's probably because "RequestContainer" is an ExtensionClass
> and ExtensionClasses are not yet (only from Zope 2.8 on) GC aware.
>
>> ...
>> Also I
>> understand zope.org suffers from this? :)
>
> Leaks can have innumerable causes. It is by no means clear
> that your leak and the one on Zope.org has the same cause.
>
>> Based on this information also, I have no reason to believe this is
>> caused by product code ... I have to guess it's a Zope bug?
>
> I do not observe leaks in my Zope installation -- even when
> I make an error stress test.
>
> This does not mean it could not be a Zope bug --
> but it implies that such a bug does not surface regularly, at most
> under special conditions.
>
>> In some ways this is motivating me to reduce the number of errors that
>> could occur, but because ALL errors seems to cause the problem, I
>> ultimately have no control over it and under certain circumstances I
>> suffer heavily from this.  404's alone probably cause this (I'm plannign
>> to test that explicitely as well).
>
> It does not here....
>
> Something is special with your installation -- maybe "Localizer" as
> Tres suggested, maybe another misbehaving product.