Zope, setlocale(), requests and traversal
Hello, I know there is a locale setting in etc/zope.conf, and I make use of it. The problem now, is that I have a multilingual site that I would like to have support multiple locales, per request: One request could be for the french site and content, using the fr_CA locale in my case, but another could be for english, and en_CA ... Localization is done through a traversal hook. So right now I call setlocale() on a per request basis, and it works fine in development. So I'm wondering: - I see the docs mention setlocale might not be threadsafe, since Zope is threaded, anyone know the implications? - If I call setlocale() during traversal, could that setting affect other threads/requests? - If the above 2 are indeed problems, is there some nice way to do set per-request locales, short of completely implementing a custom way of doing it? Thanks! Jean-François Doyon Internet Service Development and Systems Support / Spécialiste de dèveloppements internet et soutien technique Canada Centre for Remote Sensing/Centre Canadien de télédétection Natural Resources Canada/Ressources Naturelles Canada http://atlas.gc.ca Tel./Tél.: (613) 992-4902 Fax: (613) 947-2410
Hello, OK, I found out the issues, and came up with a solution, if any one cares :) Requirements: - A get_request patch - A REQUEST object that will have the necessary information to locaize from (language code, etc ...) How I do it: - Copy Python lib's locale.py to your product - import Globals - Force "locale emulation", I do it by raising an ImportError when importing _locale - Hardcode the dicts that contain the usual localconv() output for the locales you're interested in - Edit the localeconv() emulation to get the request, and from there get the localization information somehow (I base it on HTTP_ACCEPT_LANGUAGE for example), and then return the dict information. - import locale in __init__.py Works great for me, and was pretty straightforward. Notes: - You might be able to do this through monkey patching ... I copy the module because I'm using 2.5's locale module with 2.4 ... To get currency() support. - You might want to put Module security declarations in there somewhere. - Nice benefit that you can easily change the number/currency parameters to your liking, if needed. Some code (Sorry can't just share/open source it all, due to my employer's policies): <...> # Zope imports import Globals from Products.CMFCore.utils import getToolByName <...> try: # from _locale import * raise ImportError except ImportError: # Locale emulation <...> def localeconv(): """ localeconv() -> dict. Returns numeric and monetary locale-specific parameters. """ request = Globals.get_request() portal_localizer = getToolByName(request.get('PUBLISHED'), 'portal_localizer') locale_suffix = portal_localizer.getProperty('locale_suffix') return LOCALECONV[request.get('HTTP_ACCEPT_LANGUAGE', 'en') + locale_suffix] <...> LOCALECONV = {'en_CA': {'mon_decimal_point': '.', 'int_frac_digits': 2, 'p_sep_by_space': 0, 'frac_digits': 2, 'thousands_sep': ',', 'n_sign_posn': 1, 'decimal_point': '.', 'int_curr_symbol': 'CAD ', 'n_cs_precedes': 1, 'p_sign_posn': 1, 'mon_thousands_sep': ',', 'negative_sign': '-', 'currency_symbol': '$', 'n_sep_by_space': 0, 'mon_grouping': [3, 3, 0], 'p_cs_precedes': 1, 'positive_sign': '', 'grouping': [3, 3, 0]}, 'fr_CA': {'mon_decimal_point': ',', 'int_frac_digits': 2, 'p_sep_by_space': 1, 'frac_digits': 2, 'thousands_sep': ' ', 'n_sign_posn': 0, 'decimal_point': ',', 'int_curr_symbol': 'CAD ', 'n_cs_precedes': 0, 'p_sign_posn': 1, 'mon_thousands_sep': ' ', 'negative_sign': '-', 'currency_symbol': '$', 'n_sep_by_space': 1, 'mon_grouping': [3, 3, 0], 'p_cs_precedes': 0, 'positive_sign': '', 'grouping': []} } </...> Hope this comes in handy to someone! J.F. -----Original Message----- From: zope-bounces@zope.org [mailto:zope-bounces@zope.org] On Behalf Of Doyon, Jean-Francois Sent: December 14, 2006 9:23 AM To: zope@zope.org Subject: [Zope] Zope, setlocale(), requests and traversal Hello, I know there is a locale setting in etc/zope.conf, and I make use of it. The problem now, is that I have a multilingual site that I would like to have support multiple locales, per request: One request could be for the french site and content, using the fr_CA locale in my case, but another could be for english, and en_CA ... Localization is done through a traversal hook. So right now I call setlocale() on a per request basis, and it works fine in development. So I'm wondering: - I see the docs mention setlocale might not be threadsafe, since Zope is threaded, anyone know the implications? - If I call setlocale() during traversal, could that setting affect other threads/requests? - If the above 2 are indeed problems, is there some nice way to do set per-request locales, short of completely implementing a custom way of doing it? Thanks! Jean-François Doyon Internet Service Development and Systems Support / Spécialiste de dèveloppements internet et soutien technique Canada Centre for Remote Sensing/Centre Canadien de télédétection Natural Resources Canada/Ressources Naturelles Canada http://atlas.gc.ca Tel./Tél.: (613) 992-4902 Fax: (613) 947-2410 _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Doyon, Jean-Francois wrote:
- I see the docs mention setlocale might not be threadsafe, since Zope is threaded, anyone know the implications?
is setlocale a Zope or Python thing? I'm guessing the latter, which might well mean you'll get unstuck under high load here.. Zoep 3 has very good locale support, I wonder how much of that could be leveraged here? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
setlocale() is a python thing. It wraps a C library, and that function setups an application wide environment (A bit like an environment variable would ...) ... Which is why it's not thread safe. Calling setlocale() in one thread will affect the entire zope instance, across threads. Setlocale() is used by Zope to set the locale in etc/zope.conf. I found my own way to work around it (see other messages in this thread), though you're right, I didn't think of looking at the Zope 3 packages (I'm using Zope 2.10, so that should be an option for me). Maybe I'll have a look now :) J.F. -----Original Message----- From: Chris Withers [mailto:chris@simplistix.co.uk] Sent: December 15, 2006 2:24 AM To: Doyon, Jean-Francois Cc: zope@zope.org Subject: Re: [Zope] Zope, setlocale(), requests and traversal Doyon, Jean-Francois wrote:
- I see the docs mention setlocale might not be threadsafe, since Zope is threaded, anyone know the implications?
is setlocale a Zope or Python thing? I'm guessing the latter, which might well mean you'll get unstuck under high load here.. Zoep 3 has very good locale support, I wonder how much of that could be leveraged here? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Doyon, Jean-Francois wrote at 2006-12-15 08:38 -0500:
setlocale() is a python thing. It wraps a C library, and that function setups an application wide environment (A bit like an environment variable would ...) ... Which is why it's not thread safe.
If you do not have a thread aware implementation of "setlocale", then it is *VERY* dangerous to change "setlocale" inside a thread -- as changing any global data can lead to non-deterministic behaviour in a multi-threaded environment unless special precautions (locking) are taken. Apparently, you are aware of the danger. But, I have not seen how you avoid the problem in your sketched solution. -- Dieter
Well, I wanted locale.format() and locale.currency() to use a per request locale setting, instead of the underlying more "global" one that isn't thread-safe. So I patched localeconv() to extract the locale setting from the request instead of wherever it gets it otherwise normally. I've read through the locale module, and this seems to be the "deepest" point where to effect the change. This does not make the whole module thread-safe by a long shot, and wasn't intended to. I just wanted number/currency formatting to work, which it does format(), currency(), and _group() are safe). Of course now I've found the Zope 3 Locale and LocaleProvider Implementation, with number and currency and datetime support ... I may try to use that instead. J.F. -----Original Message----- From: zope-bounces@zope.org [mailto:zope-bounces@zope.org] On Behalf Of Dieter Maurer Sent: December 17, 2006 3:43 AM To: Doyon, Jean-Francois Cc: Chris Withers; zope@zope.org Subject: RE: [Zope] Zope, setlocale(), requests and traversal Doyon, Jean-Francois wrote at 2006-12-15 08:38 -0500:
setlocale() is a python thing. It wraps a C library, and that function
setups an application wide environment (A bit like an environment variable would ...) ... Which is why it's not thread safe.
If you do not have a thread aware implementation of "setlocale", then it is *VERY* dangerous to change "setlocale" inside a thread -- as changing any global data can lead to non-deterministic behaviour in a multi-threaded environment unless special precautions (locking) are taken. Apparently, you are aware of the danger. But, I have not seen how you avoid the problem in your sketched solution. -- Dieter _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
participants (3)
-
Chris Withers -
Dieter Maurer -
Doyon, Jean-Francois