April 2006 - Zope-Dev

Traversal issue which affects Five
by Alec Mitchell 21 Aug '06

21 Aug '06

Hi all, It seems that the way OFS.Traversable.restrictedTraverse() handles security checking on objects with __bobo_traverse__ methods is considerably different from the way it normally checks security. The result is that traversal cannot obtain attributes using acquisition from objects that are marked <five:traversable>. In the normal case, security is checked using guarded_getattr, which gets an attribute and checks the permissions on the retrieved object in its original context. For __bobo_traverse__methods which return simple properties (say strings), it is impossible to determine the container from which the returned attribute originates, so unless the attribute was not acquired, an Unauthorized error will always be raised. Objects that are Five Traversable always have __bobo_traverse__ methods which attempt to mimic normal traversal, which effectively means that the security checks end up preventing acquisition of simple properties using traversal from ever working on these objects (say using a TAL path expression 'context/attribute' which you expect to be acquired). Unfortunately, because Five has no control over the security checks done during traversal, this cannot be fixed directly in Five. However, IMHO fixing this makes sense for Zope itself, provided there aren't any undesirable consequences. I propose that if the validation of a __bobo_traverse result raises Unauthorized, that we make one last check to see if the result o 'guarded_getattr(obj, name)' is identical to the result of the __bobo_traverse__ call and allow it if that's the case. Here is my proposed patch against Zope 2.9 trunk: --- Traversable.py (revision 66323) +++ Traversable.py (working copy) @@ -201,9 +201,18 @@ else: # Can't determine container container = _none - if not securityManager.validate( - obj, container, name, next): - raise Unauthorized, name + try: + if not securityManager.validate( + obj, container, name, next): + raise Unauthorized, name + except Unauthorized: + # If next is a simple unwrapped property, it's + # parentage is indeterminate, but it may have been + # acquired safely. In this case validate will raise + # an error, and we can check that our value was + # acquired safely. + if guarded_getattr(obj, name, marker) is not next: + raise Unauthorized, name else: if restricted: next = guarded_getattr(obj, name, marker) At the moment Plone 2.5 is really struggling with this issue, and it would be wonderful if a fix for this could go into Zope 2.8 and 2.9 soon. I'm happy to write tests for this, I just want to make sure that I'm not proposing something really wrong/inappropriate here. Generally, the validate() call should return a True or False value, so this change should have little performance impact except in the case where 'container == _none' and validate would otherwise raise a very unhelpful unauthorized error. Your feedback is much appreciated. Alec Mitchell

6 16

favicon.ico for 2.10?
by Lennart Regebro 04 May '06

04 May '06

I find all these errors that Zope can't find a favicon.ico pretty annoying. Can we put one into 2.10s root by default, or is that for some reason a bad idea? -- Lennart Regebro, Nuxeo http://www.nuxeo.com/ CPS Content Management http://www.cps-project.org/

9 16

Zope 2.10: Remaining tasks
by Philipp von Weitershausen 02 May '06

02 May '06

The feature freeze date for the June release is only days away. It was planned to be May 1st, but due to lots of pending branches, we might let it slip for a *couple* of days (not more!). I've compiled a rough list of outstanding tasks for Zope 2.10: http://www.zope.org/Wikis/DevSite/Projects/Zope2.10/Tasks Please complete and/or correct this list. It is up to the release managers (Stephan, Andreas) and the Pope to decide the exact feature freeze date, but I would suggest that we start merging those branches real soon. I'll try to complete my work today. Philipp

7 11

Windows Test Failures and XMLExportImport
by Benji York 01 May '06

01 May '06

The BuildBot (http://buildbot.zope.org) is getting these test failures for the trunk on Windows: > Error in test test_OFS_ObjectManager__importObjectFromFile_xml (OFS.tests.test_XMLExportImport.XMLExportImportTests) > Traceback (most recent call last): > File "c:\python24\lib\unittest.py", line 260, in run > testMethod() > File "C:\buildbot\bbwin2\zc-bbwin2--Windows 2000--Zope---branches---2.9--2.4\build\lib\python\OFS\tests\test_XMLExportImport.py", line 102, in test_OFS_ObjectManager__importObjectFromFile_xml > sub._importObjectFromFile(ostream.name, 0, 0) > File "C:\buildbot\bbwin2\zc-bbwin2--Windows 2000--Zope---branches---2.9--2.4\build\lib\python\OFS\ObjectManager.py", line 602, in _importObjectFromFile > ob=connection.importFile( > File "C:\buildbot\bbwin2\zc-bbwin2--Windows 2000--Zope---branches---2.9--2.4\build\lib\python\ZODB\ExportImport.py", line 59, in importFile > f = open(f, 'rb') > IOError: [Errno 13] Permission denied: 'c:\\docume~1\\buildbot\\locals~1\\temp\\tmpftc3py.xml' > > > test_export_import_as_file_idempotent (OFS.tests.test_XMLExportImport.XMLExportImportTests) > > Error in test test_export_import_as_file_idempotent (OFS.tests.test_XMLExportImport.XMLExportImportTests) > Traceback (most recent call last): > File "c:\python24\lib\unittest.py", line 260, in run > testMethod() > File "C:\buildbot\bbwin2\zc-bbwin2--Windows 2000--Zope---branches---2.9--2.4\build\lib\python\OFS\tests\test_XMLExportImport.py", line 75, in test_export_import_as_file_idempotent > newobj = importXML(connection, ostream.name) > File "C:\buildbot\bbwin2\zc-bbwin2--Windows 2000--Zope---branches---2.9--2.4\build\lib\python\OFS\XMLExportImport.py", line 101, in importXML > file=open(file, 'rb') > IOError: [Errno 13] Permission denied: 'c:\\docume~1\\buildbot\\locals~1\\temp\\tmpizfhqn.xml' -- Benji York Senior Software Engineer Zope Corporation

2 3

Help with two strange errors.
by Lennart Regebro 01 May '06

01 May '06

I'm pretty much finished with my WSGI-implementation for Zope2, and everything works just fine, even with twisted. Except that I get one highly bizarre test failure in PythonScripts! In my branch (but not in trunk) the test that you can't import certain modules in PythonScripts fails. It makes a python script that imports mmap. This fails in my branch but, and here is the bizarre part, only if you have twisted installed. Yeah, really. Any insight into this would be helpful... Also, the ZServer test seems to leave cruft behind. Possibly a thread, that the gets detected by the zope.server tests, because it will complain that a thread is left behind by those test, while if I skip the ZServer tests with -t '!ZServer' it works fine. Does anybody with more experience of those things have any good idea of how to debug that? -- Lennart Regebro, Nuxeo http://www.nuxeo.com/ CPS Content Management http://www.cps-project.org/

1 1

request.debug needed for Zope 3 ZPT engine
by Philipp von Weitershausen 30 Apr '06

30 Apr '06

Several components that we'd like to use in Zope2, such as the SequenceEngine from Zope3, use the Zope 3 ZPT engine to render some things. (This has nothing to do with Zope 2 using or not using the Zope 3 ZPT engine.) The Zope 3 ZPT engine, in particular zope.app.pagetemplate.ViewPageTemplateFile, makes the assumption that 'request' properly implements IBrowserRequest. The Zope 2 request doesn't implement all of it, but it's close enough to make most of the Zope 3 stuff work. ViewPageTemplateFile, however, uses the 'debug' property of the request (it's specified in the request interface). This 'debug' property is an instance of DebugFlags and basically is a debugging switch for ZPT and other things. The Zope 2 request obviously doesn't have this 'debug' property. In the past we've usually added the necessary properties to the Zope 2 request in such cases. We could do this again, however: The Zope 2 request allows form data access through the __getattr__ protocol (the Zope 3 request does this through __getitem__). request.debug would shadow a potential 'debug' form variable and might break applications that make use of this form variable (I would expect there are at least a couple). Note that the obvious workaround is to use request.form['debug'] or request.cookies['debug'], depending on where you get the variable from. I would even say that this should be the preferred way of getting to form variables or cookies. But given the antiqueness of the Zope 2 API and no truly encouraged way of accessing things, there is to be expected code that relies on request.debug being the form variable. I would still vote for introducing the change. At some point (perhaps even in Zope 2.10), we'll want to use more and more of the Zope 3 ZPT engine. This problem would reoccur then. I'm therefore going to introduce the change on the zope33-port branch, as it is necessary for getting Five 1.5 (see Five's zope33-port branch) and Zope 2 to work with Zope 3.3. Comments? Philipp

3 5

Branches finished for merging.
by Lennart Regebro 30 Apr '06

30 Apr '06

I have (at least) two branches that will be merged before The Big Freeze on monday, if all goes well, and unless somebody has any very serious objections. 1. easter-sprint_traversal-refactor This was done by me an Micheal Kerrin during last weeks Sprint here in Paris. It involves a backwards compatible refactoring of the Zope2 publishing traversal. It will for special traversing of names and for default publishing traversal now use an IBrowserPublisher adapter. Objects wishing to override the default traversal would earlier have has a special __bobo_traverse__ method. They should now instead provide an IPublishTraversal adapter (or IBrowserPublisher, which extends IPublishTraversal). For backwards compatibility, any object that does not use an adapter will get a default adapter that tries to use __bobo_traverse__. Objects wishing to provide a special default name would easlier have a special __browser_default__ method. They should now intead provide an IPublishTraversal adapter. For backwards compatibility, any object that does not use an adapter will get a default adapter that tries to use __browser_default__. I also recommend that we deprecate both __bobo_traverse__ and __browser_default__, but perhaps with a longer derecation period that usual, since this are very basic techniques used in many products. Please discuss. :) 2. regebro-wsgi_refactor The work Michael and I did last week also included an effort to finish Sidnei da Silvas publication traversal branch, that attempts to replace ZPublisher with zope.publisher and ZServer with twisted. Trying to finish that work, I grew restless with the tight integration between all the modules, and have therefore tries to pry ZPublisher and ZServer apart, but wedging WSGI in between them. The modules are not 100% separated as the ZPublisher still imports some modules from the ZServer and the other way around, but there is now a WSGI application class that should be callable from any WSGI server. I propose to check this in to make it possible to use twisted with Zope2, even though we may not have time to include the twisted server in the release. Comments? -- Lennart Regebro, Nuxeo http://www.nuxeo.com/ CPS Content Management http://www.cps-project.org/

7 9

Re: SVN: Zope/branches/easter-sprint_traversal-refactor/lib/python/ZPublisher/BaseRequest.py Now using an adapter. Two tests fail in Five, and I don't know why.
by Florent Guillaume 29 Apr '06

29 Apr '06

I know this code was just refactored and exists in stock Zope 2, but it made me see this: Lennart Regebro wrote: > +class DefaultPublishTraverse(object): [...] > + # Ensure that the object has a docstring, or that the parent > + # object has a pseudo-docstring for the object. Objects that > + # have an empty or missing docstring are not published. > + doc = getattr(subobject, '__doc__', None) > + if doc is None: > + doc = getattr(object, '%s__doc__' % name, None) This should avoid acquisition and be performed on aq_base(object) I believe. Florent -- Florent Guillaume, Nuxeo (Paris, France) Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg(a)nuxeo.com

1 0

Reminder: Feature Freeze May 1
by Jim Fulton 27 Apr '06

27 Apr '06

Don't forget that the feature freeze for the June release is May 1! That is only 10 days away! New features should be check in in a *stable* form by then. While we won't necessarily do a beta release then, anything checked in for the new release must be ready for a beta release when it is checked into the trunk. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

3 3

Re: [Zope-dev] Branches finished for merging.
by Lennart Regebro 27 Apr '06

27 Apr '06

On 4/27/06, Andreas Jung <lists(a)zopyx.com> wrote: > Getting rid of ZServer is a fine goal but as long as Twisted is much slower > (from what I've heard) ZServer has to stay (ZServer is not subject to be > deprecated at this time). ZServer will remain the default for now, and it will not be deprecated unless the community wishes so. I'm guessing the community will want to sooner or later. ;-) Twisted is slower, but how much of a difference that makes in real life applications I don't know. The server is in itself not a very big performance eater in Zope. There is of course also the option to use something else that can use WSGI, like for example zope.server, which is much cleaner than ZServer, and probably therefore easier to maintain. > > The risks is that somehow we introduce some subtle behavioural change > > that isn't tested for. > > I would love to see the stuff tested with the major frameworks (CMF 2, > Plone + CPS) before it will be merged into the trunk. Sure. > > The traversal refactoring means that the role checking is now done > > after each traversal step, instead of intermingled with it. The > > functionality should be the same, but perhaps there is some untested > > special case when it changes. We have had much larger incompatibility > > changes in earlier major releases though. > > > > The WSGI refactoring may mean that somebody using internal features of > > ZServer might need to change something. Also it means that what the > > publish_module() method expect as parameters will change. > > > > I have no problems with the traversal refactoring..that sound sane. -- Lennart Regebro, Nuxeo http://www.nuxeo.com/ CPS Content Management http://www.cps-project.org/

2 1