RE: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage caused by errors
Juan David, Indeed, thanks for the input, I've ruled out new_publish as the cause as well ... Right now, I'm at the stage where I severely suspect it's got something to do with i18n support or translation somehow, but don't know where exactly yet. If I remove the message_catalog, the problem goes away. If I remove the TranslationService instance, the problem goes away. Do you use this? Does anybody know of alternatives to the TranslationService+Localizer combo to enable i18n in ZPT's on Zope 2? If I keep the above two, but delete my custom Localizer instances, the problem persists. I was trying to find an alternative message_catalog to use, but couldn't find one (CMFLocalizer is based on yours). So I'm going to try and narrow the field some more. I *quickly* looked the message_catalog and TranslationService code, and nothing jumped out. I have to also look at the ZPT engine itself, maybe it's i18n support is the cause of the problem. I'll run further tests ... Thanks, J.F. -----Original Message----- From: "J. David Ibáñez" [mailto:jdavid@itaapy.com] Sent: May 21, 2004 8:17 AM To: Jean-Francois.Doyon@CCRS.NRCan.gc.ca Cc: dieter@handshake.de; zope-dev@zope.org Subject: Re: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage caused by errors The problem I have with one of my sites seems to have a different cause, as it does not show up when errors occur, but when requests are succesful. So far I haven't been able to reproduce the problem. You can try to modify "new_publish" to catch exceptions: def new_publish(request, module_name, after_list, debug=0): id = get_ident() Publish._requests[id] = request try: x = Publish.old_publish(request, module_name, after_list, debug) except: del Publish._requests[id] raise else: try: del Publish._requests[id] except KeyError: ... Though it is unlikely the leak to be there. Jean-Francois.Doyon@CCRS.NRCan.gc.ca wrote:
Indeed, I was hoping the ref in the dictionary there was getting in the way when old_publish was raising an exception, but I tried using a weakref.proxy() instead and that didn't help.
I managed to test without Localizer (It dawned on me that since I'm looking at errors, it really doesn't matter if I don't have Localizer installed since not having it will generate errors, which is what I want!) ... And the problem diappeared ...
So it's definitely Localizer, but it's not the Globals() request patch ... Or at least not the act of patching it itself, that causes the problem.
*sigh* J.F.
-----Original Message----- From: Dieter Maurer [mailto:dieter@handshake.de] Sent: May 20, 2004 2:58 PM To: Jean-Francois.Doyon@CCRS.NRCan.gc.ca Cc: jdavid@itaapy.com; tseaver@zope.com; zope-dev@zope.org; chrism@plope.com; brian@zope.com Subject: RE: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage caused by errors
Jean-Francois.Doyon@CCRS.NRCan.gc.ca wrote at 2004-5-20 09:58 -0400:
... def new_publish(request, module_name, after_list, debug=0): id = get_ident() print "Localizer got thread id: " + str(id) Publish._requests[id] = request print "Request dict is now: " + str(Publish._requests) x = Publish.old_publish(request, module_name, after_list, debug) try: del Publish._requests[id] except KeyError: ...
This code cannot leak requests in large numbers (though it may leak a few requests for some time).
Reason: Zope does not normally create new threads. This implies that thread_ids are reused and thereby old requests flushed from the dict.
-- J. David Ibáñez Founder and CTO of Itaapy <http://www.itaapy.com> 9 rue Darwin, 75018 Paris Tel +33 (0)1 42 23 67 45 / Fax +33 (0)1 53 28 27 88
No, I don't use TranslationService. A quick glance at the code shows that TS uses several caches (dictionaries) which are stored in the request object: "_localizer_placeful_mc_cache", "_translation_service_cache" and "_ts_domain_cache". The values of these dictionaries are acquisition wrappers (objects returned by "unrestrictedTraverse" or explicitly built). Maybe the leak is there. Jean-Francois.Doyon@CCRS.NRCan.gc.ca wrote:
Juan David,
Indeed, thanks for the input, I've ruled out new_publish as the cause as well ...
Right now, I'm at the stage where I severely suspect it's got something to do with i18n support or translation somehow, but don't know where exactly yet.
If I remove the message_catalog, the problem goes away.
If I remove the TranslationService instance, the problem goes away. Do you use this? Does anybody know of alternatives to the TranslationService+Localizer combo to enable i18n in ZPT's on Zope 2?
If I keep the above two, but delete my custom Localizer instances, the problem persists.
I was trying to find an alternative message_catalog to use, but couldn't find one (CMFLocalizer is based on yours).
So I'm going to try and narrow the field some more. I *quickly* looked the message_catalog and TranslationService code, and nothing jumped out. I have to also look at the ZPT engine itself, maybe it's i18n support is the cause of the problem.
I'll run further tests ...
Thanks, J.F.
-----Original Message----- From: "J. David Ibáñez" [mailto:jdavid@itaapy.com] Sent: May 21, 2004 8:17 AM To: Jean-Francois.Doyon@CCRS.NRCan.gc.ca Cc: dieter@handshake.de; zope-dev@zope.org Subject: Re: [Zope-dev] Re: Preliminary findings: Zope 2.7 leakage caused by errors
The problem I have with one of my sites seems to have a different cause, as it does not show up when errors occur, but when requests are succesful.
So far I haven't been able to reproduce the problem.
You can try to modify "new_publish" to catch exceptions:
def new_publish(request, module_name, after_list, debug=0): id = get_ident() Publish._requests[id] = request try: x = Publish.old_publish(request, module_name, after_list, debug) except: del Publish._requests[id] raise else: try: del Publish._requests[id] except KeyError: ...
Though it is unlikely the leak to be there.
-- J. David Ibáñez Founder and CTO of Itaapy <http://www.itaapy.com> 9 rue Darwin, 75018 Paris Tel +33 (0)1 42 23 67 45 / Fax +33 (0)1 53 28 27 88
J. David Ibanez wrote at 2004-5-21 16:50 +0200:
No, I don't use TranslationService. A quick glance at the code shows that TS uses several caches (dictionaries) which are stored in the request object: "_localizer_placeful_mc_cache", "_translation_service_cache" and "_ts_domain_cache".
The values of these dictionaries are acquisition wrappers (objects returned by "unrestrictedTraverse" or explicitly built). Maybe the leak is there.
When these dictionaries are held in "other" (or "form" or "Lazy"), then there is no problem -- as "other" is explicitly clear when the request is closed. However, when they are (direct) attributes of the request object, then they will form cycles which almost surely cause leakage. -- Dieter
Jean-Francois.Doyon@CCRS.NRCan.gc.ca wrote at 2004-5-21 09:34 -0400:
... If I remove the TranslationService instance, the problem goes away. Do you use this? Does anybody know of alternatives to the TranslationService+Localizer combo to enable i18n in ZPT's on Zope 2?
I am no internationalization expert. But many posts suggest to use "PTS" (Placeless Translation Service) nowadays... -- Dieter
participants (3)
-
Dieter Maurer -
J. David Ibanez -
Jean-Francois.Doyon@CCRS.NRCan.gc.ca