[Zope3-checkins] SVN: Zope3/trunk/ - Fixed bug #738: RestrictedPython was unable to parse$ Unicode expressions

[Christian Theune]

Log message for revision 71527:
  - Fixed bug #738: RestrictedPython was unable to parse$ Unicode expressions
    correctly (as passed in from e.g. ZPTPages).$
  
  

Changed:
  U   Zope3/trunk/doc/CHANGES.txt
  U   Zope3/trunk/src/RestrictedPython/RCompile.py
  A   Zope3/trunk/src/RestrictedPython/tests/testCompile.py
  U   Zope3/trunk/src/RestrictedPython/tests/testRestrictions.py

-=-
Modified: Zope3/trunk/doc/CHANGES.txt
===================================================================
--- Zope3/trunk/doc/CHANGES.txt	2006-12-11 17:12:54 UTC (rev 71526)
+++ Zope3/trunk/doc/CHANGES.txt	2006-12-11 17:27:06 UTC (rev 71527)
@@ -149,6 +149,9 @@
 
     Bug fixes
 
+      - Fixed bug #738: RestrictedPython was unable to parse
+        Unicode expressions correctly (as passed in from e.g. ZPTPages).
+
       - Fixed bug #723: Testbrowser was handling multiple submit buttons with
         the same name incorrectly.
 

Modified: Zope3/trunk/src/RestrictedPython/RCompile.py
===================================================================
--- Zope3/trunk/src/RestrictedPython/RCompile.py	2006-12-11 17:12:54 UTC (rev 71526)
+++ Zope3/trunk/src/RestrictedPython/RCompile.py	2006-12-11 17:27:06 UTC (rev 71527)
@@ -25,6 +25,10 @@
 
 
 def niceParse(source, filename, mode):
+    if isinstance(source, unicode):
+        # Use the utf-8-sig BOM so the compiler
+        # detects this as a UTF-8 encoded string.
+        source = '\xef\xbb\xbf' + source.encode('utf-8')
     try:
         return parse(source, mode)
     except:

Added: Zope3/trunk/src/RestrictedPython/tests/testCompile.py
===================================================================
--- Zope3/trunk/src/RestrictedPython/tests/testCompile.py	2006-12-11 17:12:54 UTC (rev 71526)
+++ Zope3/trunk/src/RestrictedPython/tests/testCompile.py	2006-12-11 17:27:06 UTC (rev 71527)
@@ -0,0 +1,36 @@
+# -*- coding: utf-8 -*-
+##############################################################################
+#
+# Copyright (c) 2006 Zope Corporation and Contributors. All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE
+#
+##############################################################################
+__version__ = '$Revision$'[11:-2]
+
+import unittest
+
+from RestrictedPython.RCompile import niceParse
+import compiler.ast
+
+class CompileTests(unittest.TestCase):
+
+    def testUnicodeSource(self):
+        # We support unicode sourcecode.
+        source = u"u'Ä väry nice säntänce with umlauts.'"
+
+        parsed = niceParse(source, "test.py", "exec")
+        self.failUnless(isinstance(parsed, compiler.ast.Module))
+        parsed = niceParse(source, "test.py", "single")
+        self.failUnless(isinstance(parsed, compiler.ast.Module))
+        parsed = niceParse(source, "test.py", "eval")
+        self.failUnless(isinstance(parsed, compiler.ast.Expression))
+
+
+def test_suite():
+    return unittest.makeSuite(CompileTests)


Property changes on: Zope3/trunk/src/RestrictedPython/tests/testCompile.py
___________________________________________________________________
Name: svn:keywords
   + Id Rev Date
Name: svn:eol-style
   + native

Modified: Zope3/trunk/src/RestrictedPython/tests/testRestrictions.py
===================================================================
--- Zope3/trunk/src/RestrictedPython/tests/testRestrictions.py	2006-12-11 17:12:54 UTC (rev 71526)
+++ Zope3/trunk/src/RestrictedPython/tests/testRestrictions.py	2006-12-11 17:27:06 UTC (rev 71527)
@@ -448,6 +448,7 @@
 
 create_rmodule()
 
+
 def test_suite():
     return unittest.makeSuite(RestrictionTests, 'check')