UTF encoding problem w/ ZPT's (No, not the usual :)
G'day, So I've been working on this project for some time, and all is well. Now for the first time I migreate my product code to a brand new Zope installation on the production systems. Development and production systems are as identical as they can be. In this case, they both have the same default locale (as indicated by the locale command in the shell), same OS version, same zope.conf settings (default-zpublisher-encoding and locale), etc ... I use UTF-8 *everywhere* (My PT files, .py files, etc ... all UTF-8). The web pages come out UTF-8 ... I've even switched the ZMI to UTF-8. All works well on the development side. Now on the production side, I'm getting an encoding related error. Where I have something like: <meta name="description" xml:lang="fra" content="..." i18n:attributes="content" i18n:target="string:fr" /> The value of i18n:target 'string:fr' has always been a str from the beginning. Now, on the new system, it comes through as a unicode string! So I see this: Error Type: TypeError Error Value: 'unicode' object is not callable Since the i18n:target implementation is my own, I can fix the code of course, but now I'm worried this might happen somewhere else, and I don't understand why. The only difference I can see is that the file went through subversion in the process (commit -> check out on different machine). Anyone know where this difference might come from? In other words, how does the PT engine decide whether that value shuld be a unicode string or a regular one? Any insight would be much appreciated :) Thanks, Jean-François Doyon Data Dissemination Division | Division de la diffusion des données Data Management and Dissemination Branch | Direction de la gestion et de la diffusion des données Earth Sciences Sector | Secteur des sciences de la Terre Natural Resources Canada | Ressources naturelles Canada Ottawa, Canada K1A 0E9 jdoyon@nrcan-rncan.gc.ca Telephone | Téléphone 613-992-4902 Facsimile | Télécopieur 613-947-2410 Teletypewriter | Téléimprimeur 613-996-4397 Government of Canada | Gouvernement du Canada
Doyon, Jean-Francois wrote:
So I see this:
Error Type: TypeError Error Value: 'unicode' object is not callable
Usual comments, full traceback please. This doesn't look like a normal encoding error but a bug in code somewhere...
The only difference I can see is that the file went through subversion in the process (commit -> check out on different machine).
Subversion will likely screw with the encoding ;-) cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Ooops, my bad ... Here it is: Traceback (innermost last): Module ZPublisher.Publish, line 119, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 42, in call_object Module Shared.DC.Scripts.Bindings, line 313, in __call__ Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec Module Products.CMFCore.FSPageTemplate, line 216, in _exec Module Products.CMFCore.FSPageTemplate, line 155, in pt_render Module Products.PageTemplates.PageTemplate, line 89, in pt_render Module zope.pagetemplate.pagetemplate, line 117, in pt_render Module zope.tal.talinterpreter, line 271, in __call__ Module zope.tal.talinterpreter, line 346, in interpret Module zope.tal.talinterpreter, line 379, in do_startEndTag Module zope.tal.talinterpreter, line 408, in do_startTag Module zope.tal.talinterpreter, line 496, in attrAction_tal Module Products.ESSPortals.i18ntargetpatch, line 34, in talinterpretertranslate Module Products.ESSPortals.i18ntargetpatch, line 14, in zopecontexttranslate Module Products.PageTemplates.Expressions, line 224, in evaluateText Module zope.tales.tales, line 696, in evaluate - URL: file:/home/atlas/app/Zope-CPS-Instance/Products/ESSPortals/skins/atlas/s plash/root_view.pt - Line 33, Column 0 - Expression: u'string:fr' - Names: {'container': <ESSSite at /atlas>, 'context': <ESSSite at /atlas>, 'default': <object object at 0xb7fe3528>, 'here': <ESSSite at /atlas>, 'loop': {}, 'nothing': None, 'options': {'args': ()}, 'repeat': <Products.PageTemplates.Expressions.SafeMapping object at 0xb364d5cc>, 'request': <HTTPRequest, URL=http://atlas.nrcan.gc.ca/new/root_view>, 'root': <Application at >, 'template': <FSPageTemplate at /atlas/root_view>, 'traverse_subpath': [], 'user': <SpecialUser 'Anonymous User'>} TypeError: 'unicode' object is not callable The problem is that zope.tales.tales.evaluate() tests for a str ... But, now, for unknown reasons, it's getting a unicode string. The i18ntarget patch code does nothing to the value of the i18n:target attribute. Up until yesterday, target_language was always a string ... Now, on the new servers, it's a unicode string! (And like I said, the environments are otherwise quite identical). Here's the monkey patch to enable i18n:target support: # $Id: i18ntargetpatch.py 2481 2007-08-29 17:24:02Z jdoyon $ """ Make i18n:target function in ZPT's. """ from Products.PageTemplates.Expressions import ZopeContext from Products.PageTemplates.GlobalTranslationService import \ getGlobalTranslationService from zope.tal.talinterpreter import TALInterpreter def zopecontexttranslate(self, msgid, domain=None, mapping=None, target_language=None, default=None): context = self.contexts.get('context') if target_language: target_language = self.evaluateText(target_language) return getGlobalTranslationService().translate( domain, msgid, mapping=mapping, context=context, target_language=target_language, default=default) ZopeContext.translate = zopecontexttranslate def talinterpretertranslate(self, msgid, default=None, i18ndict=None, obj=None, domain=None): if default is None: default = getattr(msgid, 'default', unicode(msgid)) if i18ndict is None: i18ndict = {} if domain is None: domain = getattr(msgid, 'domain', self.i18nContext.domain) if obj: i18ndict.update(obj) if not self.i18nInterpolate: return msgid return self.engine.translate(msgid, self.i18nContext.domain, i18ndict, self.i18nContext.target, default) TALInterpreter.translate = talinterpretertranslate We'll see with time if this crops up anywhere else ... Though I use i18n:target in very few places, and don't usually otherwise handle attribute values directly the way I do here. As for subversion, I took great care to make sure everything was utf-8 on the FS, during development, and I know Subersion uses utf-8 internally, so I'm not sure what it would change it to. Once checked back out, it still looks like utf-8 ... I'm saying all this based on the assumption that the string -> unicode difference might come from the encoding of the PT itself? J.F. -----Original Message----- From: zope-bounces@zope.org [mailto:zope-bounces@zope.org] On Behalf Of Chris Withers Sent: August 30, 2007 06:45 To: Doyon, Jean-Francois Cc: zope@zope.org Subject: Re: [Zope] UTF encoding problem w/ ZPT's (No, not the usual :) Doyon, Jean-Francois wrote:
So I see this:
Error Type: TypeError Error Value: 'unicode' object is not callable
Usual comments, full traceback please. This doesn't look like a normal encoding error but a bug in code somewhere...
The only difference I can see is that the file went through subversion in the process (commit -> check out on different machine).
Subversion will likely screw with the encoding ;-) cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
As for subversion, I took great care to make sure everything was utf-8 on the FS, during development, and I know Subersion uses utf-8 internally, so I'm not sure what it would change it to. Once checked back out, it still looks like utf-8 ... I'm saying all this based on the assumption that the string -> unicode difference might come from the encoding of the PT itself?
Which Zope version do you have and are you sure your versions are same in both environments? -- Maciej Wisniowski
Yes, it's Zope 2.10.4 with CMF 2.1.0, all of them on Python 2.4.4, all of them on CentOS 4.4 (or maybe .5 now?) (maybe not at exactly the same patch/upgrade level, but the python is custom built). In fact the entire product list is exactly the same! OK, so I think maybe my devel machine is CentOS 4.4, and the production systems are CentOS 4.5, but that's as much difference as there might be, worst case scenario. I have a hard time picturing this making much of a difference in the PT encoding though :P *scratches head* J.F. -----Original Message----- From: Maciej Wisniowski [mailto:maciej.wisniowski@coig.katowice.pl] Sent: August 30, 2007 08:49 To: Doyon, Jean-Francois Cc: Chris Withers; zope@zope.org Subject: Re: [Zope] UTF encoding problem w/ ZPT's (No, not the usual :)
As for subversion, I took great care to make sure everything was utf-8
on the FS, during development, and I know Subersion uses utf-8 internally, so I'm not sure what it would change it to. Once checked back out, it still looks like utf-8 ... I'm saying all this based on the assumption that the string -> unicode difference might come from the encoding of the PT itself?
Which Zope version do you have and are you sure your versions are same in both environments? -- Maciej Wisniowski
Doyon, Jean-Francois wrote:
'user': <SpecialUser 'Anonymous User'>} TypeError: 'unicode' object is not callable
The problem is that zope.tales.tales.evaluate() tests for a str ... But, now, for unknown reasons, it's getting a unicode string.
...but even if it got an str, str's aren't callable either. Might be something to look at... cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Well it tries to call only because the varial ISN'T a str :) If it's a string it behaves one way, else, it behaves another, and tries to call what it got. I imagine there's circumstances where this is used. I'll put something in the collector then, and we'll see where it goes. J.F. -----Original Message----- From: zope-bounces@zope.org [mailto:zope-bounces@zope.org] On Behalf Of Chris Withers Sent: September 3, 2007 02:55 To: Doyon, Jean-Francois Cc: zope@zope.org Subject: Re: [Zope] UTF encoding problem w/ ZPT's (No, not the usual :) Doyon, Jean-Francois wrote:
'user': <SpecialUser 'Anonymous User'>} TypeError: 'unicode' object is not callable
The problem is that zope.tales.tales.evaluate() tests for a str ... But, now, for unknown reasons, it's getting a unicode string.
...but even if it got an str, str's aren't callable either. Might be something to look at... cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
participants (3)
-
Chris Withers -
Doyon, Jean-Francois -
Maciej Wisniowski