[Zope-Checkins] CVS: Zope/lib/python/TreeDisplay - TreeTag.py:1.53.68.4

Tres Seaver tseaver at zope.com
Thu Jan 8 18:34:30 EST 2004


Update of /cvs-repository/Zope/lib/python/TreeDisplay
In directory cvs.zope.org:/tmp/cvs-serv30073/lib/python/TreeDisplay

Modified Files:
      Tag: Zope-2_7-branch
	TreeTag.py 
Log Message:


  Merge security audit work for the 2.7 branch:

    - Collector #1140: setting the access control implementation from
      the configuration file didn't work.  The ZOPE_SECURITY_POLICY
      environment variable is no longer honored.

    - Browsers that do not escape html in query strings such as 
      Internet Explorer 5.5 could potentially send a script tag in a 
      query string to the ZSearch interface for cross-site scripting.

    - FilteredSets (used within TopicIndex) are defined via an expression,
      which was naievely eval'ed.

    - The ZTUtils SimpleTree decompressed tree state data from the 
      request without checking for final size, which could allow for 
      certain types of DoS attacks.

    - Inadequate security assertions on administrative "find" methods 
      could potentially be abused.

    - Some improper security assertions on DTMLDocument objects could 
      potentially allow access to members that should be protected.

    - Class security was not properly intialized for PythonScripts, 
      potentially allowing access to variables that should be protected. 
      It turned out that most of the security assertions were in fact 
      activated as a side effect of other code, but this fix is still 
      appropriate to ensure that all security declarations are properly 
      applied.

    - The dtml-tree tag used an "eval" of user-supplied data; its 
      efforts to prevent abuse were ineffective.

    - XML-RPC marshalling of class instances used the instance 
      __dict__ to marshal the object, and could include attributes 
      prefixed with an underscore name. These attributes are considered 
      private in Zope and should generally not be disclosed.

    - Some property types were stored in a mutable data type (list) which 
      could potentially allow untrusted code to effect changes on those 
      properties without going through appropriate security checks in 
      particular scenarios.

    - Inadequate type checking could allow unicode values passed to 
      RESPONSE.write() to be passed into deeper layers of asyncore, 
      where an exception would eventually be generated at a level that 
      would cause the Zserver main loop to terminate.

    - The variables bound to page templates and Python scripts such as 
      "context" and "container" were not checked adequately, allowing 
      a script to potentially access those objects without ensuring the 
      necessary permissions on the part of the executing user.

    - Iteration over sequences could in some cases fail to check access 
      to an object obtained from the sequence. Subsequent checks (such 
      as for attributes access) of such an object would still be 
      performed, but it should not have been possible to obtain the 
      object in the first place.

    - List and dictionary instance methods such as the get method of 
      dictionary objects were not security aware and could return an 
      object without checking access to that object. Subsequent checks 
      (such as for attributes access) of such an object would still be 
      performed, but it should not have been possible to obtain the 
      object in the first place.

    - Use of 'import as. in Python scripts could potentially rebind 
      names in ways that could be used to avoid appropriate security 
      checks.

    - A number of newer built-ins (min, max, enumerate, iter, sum)
      were either unavailable in untrusted code or did not perform
      adequate security checking.

    - Unpacking via function calls, variable assignment, exception 
      variables and other contexts did not perform adequate security 
      checks, potentially allowing access to objects that should have 
      been protected.

    - DTMLMethods with proxy rights could incorrectly transfer those 
      rights via acquisition when traversing to a parent object.



=== Zope/lib/python/TreeDisplay/TreeTag.py 1.53.68.3 => 1.53.68.4 ===
--- Zope/lib/python/TreeDisplay/TreeTag.py:1.53.68.3	Thu Dec 11 17:51:08 2003
+++ Zope/lib/python/TreeDisplay/TreeTag.py	Thu Jan  8 18:33:59 2004
@@ -18,9 +18,10 @@
 from DocumentTemplate.DT_Util import *
 from DocumentTemplate.DT_String import String
 
+from cPickle import dumps
 from string import translate
 from urllib import quote, unquote
-from zlib import compress, decompress
+from zlib import compress, decompressobj
 from binascii import b2a_base64, a2b_base64
 import re
 
@@ -115,15 +116,15 @@
       ['eagle'], # eagle is open
       ['eagle'], ['jeep', [1983, 1985]]  # eagle, jeep, 1983 jeep and 1985 jeep
 
-    where the items are object ids. The state will be converted to a
+    where the items are object ids. The state will be pickled to a
     compressed and base64ed string that gets unencoded, uncompressed,
-    and evaluated on the other side.
+    and unpickled on the other side.
 
     Note that ids used in state need not be connected to urls, since
     state manipulation is internal to rendering logic.
 
-    Note that to make eval safe, we do not allow the character '*' in
-    the state.
+    Note that to make unpickling safe, we use the MiniPickle module,
+    that only creates safe objects
     """
 
     data=[]
@@ -333,7 +334,7 @@
 
             ####################################
             # Mostly inline encode_seq for speed
-            s=compress(str(diff))
+            s=compress(dumps(diff,1))
             if len(s) > 57: s=encode_str(s)
             else:
                 s=b2a_base64(s)[:-1]
@@ -523,7 +524,7 @@
 
 def encode_seq(state):
     "Convert a sequence to an encoded string"
-    state=compress(str(state))
+    state=compress(dumps(state))
     l=len(state)
 
     if l > 57:
@@ -583,10 +584,22 @@
         state=a2b_base64(state)
 
     state=decompress(state)
-    if state.find('*') >= 0: raise ValueError, 'Illegal State: %s' % state
-    try: return list(eval(state,{'__builtins__':{}}))
-    except: return []
-
+    try:
+        return list(MiniUnpickler(StringIO(state)).load())
+    except:
+        return []
+
+def decompress(input,max_size=10240):
+    # This sillyness can go away in python 2.2
+    d = decompressobj()
+    output = ''
+    while input:
+        fragment_size = max(1,(max_size-len(output))/1000)
+        fragment,input = input[:fragment_size],input[fragment_size:]
+        output += d.decompress(fragment)
+        if len(output)>max_size:
+            raise ValueError('Compressed input too large')
+    return output+d.flush()
 
 def tpStateLevel(state, level=0):
     for sub in state:
@@ -632,3 +645,79 @@
 #icoSpace='<IMG SRC="Blank_icon" BORDER="0">'
 #icoPlus ='<IMG SRC="Plus_icon" BORDER="0">'
 #icoMinus='<IMG SRC="Minus_icon" BORDER="0">'
+
+
+
+
+
+###############################################################################
+## Everthing below here should go in a MiniPickle.py module, but keeping it
+## internal makes an easier patch
+
+
+import pickle
+from cStringIO import StringIO
+
+
+if pickle.format_version!="2.0":
+    # Maybe the format changed, and opened a security hole
+    raise 'Invalid pickle version'
+
+
+class MiniUnpickler(pickle.Unpickler):
+    """An unpickler that can only handle simple types.
+    """
+    def refuse_to_unpickle(self):
+        raise pickle.UnpicklingError, 'Refused'
+
+    dispatch = pickle.Unpickler.dispatch.copy()
+
+    for k,v in dispatch.items():
+        if k=='' or k in '().012FGIJKLMNTUVX]adeghjlpqrstu}':
+            # This key is necessary and safe, so leave it in the map
+            pass
+        else:
+            dispatch[k] = refuse_to_unpickle
+            # Anything unnecessary is banned, but here is some logic to explain why
+            if k in [pickle.GLOBAL, pickle.OBJ, pickle.INST, pickle.REDUCE, pickle.BUILD]:
+                # These are definite security holes
+                pass
+            elif k in [pickle.PERSID, pickle.BINPERSID]:
+                # These are just unnecessary
+                pass
+            elif k in [pickle.STRING]:
+                # This one is controversial: A string is harmlessm, but the
+                # implementation of pickle leaks memory (strings may be interned)
+                # The problem can be avoided by using binary pickles.
+                pass
+    del k
+    del v
+
+def _should_succeed(x,binary=1):
+    if x != MiniUnpickler(StringIO(pickle.dumps(x,binary))).load():
+        raise ValueError(x)
+
+def _should_fail(x,binary=1):
+    try:
+        MiniUnpickler(StringIO(pickle.dumps(x,binary))).load()
+        raise ValueError(x)
+    except pickle.UnpicklingError, e:
+        if e[0]!='Refused': raise ValueError(x)
+
+class _junk_class: pass
+
+def _test():
+    _should_fail('hello',0)
+    _should_succeed('hello')
+    _should_succeed(1)
+    _should_succeed(1L)
+    _should_succeed(1.0)
+    _should_succeed((1,2,3))
+    _should_succeed([1,2,3])
+    _should_succeed({1:2,3:4})
+    _should_fail(open)
+    _should_fail(_junk_class)
+    _should_fail(_junk_class())
+
+# Test MiniPickle on every import
+_test()




More information about the Zope-Checkins mailing list