December 2011 - Zope-Dev

Zope 4 release management
by Laurence Rowe 12 Jun '12

12 Jun '12

Along with David Glick, I would like to volunteer for the Zope 4 release management role, where I would take responsibility for producing the initial release of Zope 4 and David would then take over for the maintenance releases. In doing so, I thought it would be helpful to set out our understanding of the mission of Zope 4. If accepted I'll work to produce a first draft of a roadmap based on the outcome of the recent sprints and discussions on this mailing list. Mission ------- Zope 4 will be the first of several releases that seek to simplify our software stack. Over the years Zope 2 grew through the adoption of new technologies, notably Zope 3, but rarely removed older ways of doing things. Below, 'Zope 4' refers to the series of releases including Zope 5, 6, etc. Over the series of releases, Zope 4 will progressively remove more and more software as we transition to using software components shared with other Python web frameworks. It is as important to state what Zope 4 *is not* as what it should be: * Zope 4 will not seek to be of interest to those developing new applications. They should instead look to projects such as Pyramid that build on the experience and technologies of Zope without regard for the backwards compatibility concerns that will constrain Zope 4. * Zope 4 will not seek to innovate in itself but encourage innovation in software components shared with the wider Python web community. * Zope 4 will not move so quickly that it becomes impossible to make Plone, its main consumer, work with it. * But neither will Zope 4 seek to retain backwards compatibility at any cost. As the basis of Plone 4, Zope 2.13 will see maintenance releases as long as Plone 4 is supported. * Zope 4 will not have a release cycle independent of Plone. Zope 4 only exists as a transitional path for Zope 2 based applications and experience has shown that Zope 2 releases not used in any Plone release do not receive the same level of ongoing maintenance. Development Process ------------------- We want to encourage all features to be developed on separate feature branches so we can maintain a stable trunk. Before these feature branches are merged they should be posted to the mailing list for review. This process will necessitate a lot of merging, so I want to propose that we move to Git for development (something we found very helpful at our recent San Francisco Zope 4 sprint.) I don't have any opinion on where the canonical repository should be hosted - I know some people have strong opinions that this should be on Zope Foundation controlled hardware. If that's to be the case then we will need the svn.zope.org maintainers to setup a shared git repository on that machine. I think mirroring to github (and/or launchpad in future) will be the simplest way to provide an anonymously accessible and web browsable copy. Laurence

17 32

Re: [Zope-dev] experimental.broken - Graceful handling of broken interfaces and components in the ZODB
by Ross Patterson 11 Apr '12

11 Apr '12

I've done some more work on this and I've gotten the component registrations fully working now with one exception that I'm having real trouble with. I'd like some help with that, more below. I'm also a bit more clear on what might be appropriate to bring back into zope.interface and I'd like feedback on that. Currently interfaces are pickled as globals. Given their centrality in any ZCA application, I think they should be pickled using a function: def __reduce__(self): return (find_interface, (modulename, globalname)) where find_interface, if ZODB.broken.find_global can be imported, in turn calls: ZODB.broken.find_global(modulename, globalname, Broken=IBroken, type=InterfaceClass) since find_global already has nicely abstracted graceful missing global handling. If this were added to zope.interface, and changed ZODB objects with marker interfaces or persistent registries where all the code for the interfaces is still available will then be updated with pickles that will gracefully handle missing interfaces in the future. Thus you could use zodbupdate to make sure that the interfaces in your ZODB will fail gracefully in the future. Do you agree/disagree that this belongs in zope.interface? What this will not address are existing interfaces-as-globals pickles where the original interface is now missing. That's where the other patches in experimental.broken come in, they intercept the two contexts where we can infer that a missing global should be an interface: instance provides declarations and persistent component registries. By hooking into the unpickling of those objects, we can replace broken classes with broken interfaces as appropriate for those structures. With these patches installed it should be possible to use applications with missing interfaces and to use zodbupdate to make sure that even *already missing* interfaces will fail gracefully both now and in the future. I think these patches don't belong in zope.interface/component and if they work I would likely move them to five.localsitemanager along with a ZCML file that is not loaded by default. Does that sound right? Then one thing I haven't been able to get working is making it possible to commit a changed persistent registry when it includes a component registration for a non-persistent component whose class/type is missing. This works just fine for a *persistent* component whose class/type is missing but fails for a non-persistent component. The error is: AttributeError: 'Bar' object has no attribute '__Broken_newargs__' More specifically, '__Broken_newargs__' is set by the Broken.__new__ method and I've confirmed that this isn't being called by instrumenting __new__, __init__, and __setstate__. Only __setstate__ is called when unpickling the non-persistent broken component not __new__ as should be. Below is the full traceback. I've also left the package repo in the broken state if you want to examine it, the egg checkout is also a buildout: https://github.com/rpatterson/experimental.broken/commit/a4b648dc78e53c7303… Any help with this last issue would be appreciated. Ross Ross Patterson <me(a)rpatterson.net> writes: > Please take a look at experimental.broken: > > https://github.com/rpatterson/experimental.broken > http://pypi.python.org/pypi/experimental.broken > > The handling of broken objects by the ZODB can make an application with > add-ons that use zope.interface far too fragile. If marker interfaces > from an add-on are used on objects in the ZODB, removing that add-on can > make any zope.interface operation on that object fail. Even worse, if > an add-on registers any components in a registry in the ZODB, that > entire registry will become unusable for any ZCA operations which pretty > much breaks everything, including admin interfaces. > > Since the interfaces and the ZCA are often core parts of an application > using the ZODB, it may be appropriate to add special handling for broken > objects to those services. The experimental.broken patches are my > attempt to prototype such special handling. > > For objects in the ZODB which directly provide a marker interface, > these patches allow that object to behave as without the application > of the marker interface if the interface is no longer available. If > the interface is made available again, the full behavior of that > interface is restored. Similarly, if a component whose class, > provided interface, or required interfaces are missing, these patches > allow the registry to perform lookups it would have been able to do > without the broken component registration. If the component > class, provided interface, and required interfaces are restored, > then the component registration is fully restored. > > If an object or registry in the ZODB is committed to the ZODB with > broken interfaces or components, the commit will succeed and it is still > possible to fully restore previous behavior if the missing classes and > interfaces are restored. Unfortunately, because interfaces are pickled > as globals, there's no good way to have the same pickle written on > commit for the interface as was in the original pickle, but it should > behave exactly the same. > > The intention of this package is to see if the implementation of broken > object handling is correct and robust enough to merge into > zope.interface and zope.component themselves. Is this the right > approach? If not why and what would be better? How might this approach > be improved? > > Ross > > ------------------------------------------------------------------------------ > RSA(R) Conference 2012 > Save $700 by Nov 18 > Register now > http://p.sf.net/sfu/rsa-sfdev2dev1

7 10

Re: [Zope-dev] [Plone-developers] experimental.broken - Graceful handling of broken interfaces and components in the ZODB
by Malthe Borch 08 Apr '12

08 Apr '12

On 7 November 2011 09:17, Ross Patterson <me(a)rpatterson.net> wrote: > The intention of this package is to see if the implementation of broken > object handling is correct and robust enough to merge into > zope.interface and zope.component themselves. Is this the right > approach? If not why and what would be better? How might this approach > be improved? (removed plone-dev from cc). Isn't it symptom treatment though? If you've got an add-on which adds marker interfaces to "general objects", shouldn't that add-on remove – or no longer provide – those same interfaces when it's uninstalled? At least in Plone, you can easily query content objects providing a particular set of interfaces. I think it's a non-goal to be able to run a system without all the required software – which is how I understand it when you just do a "hard remove" of an add-on without a prior "soft remove". \malthe

3 2

zope.server still used?
by Chris McDonough 10 Jan '12

10 Jan '12

Hi all, Is zope.server still largely used by e.g. bluebream, grok, and other zope 3 apps? Or do people tend to use other WSGI servers instead? Either way, arguments for or against zope.server would be useful; I'm trying to decide whether to base some new stuff on it. - C

6 7

Zope internals documentation
by Martin Aspeli 02 Jan '12

02 Jan '12

Hi folks, I have documented some of the darker corners of Zope's internals. I put it in the Plone developer documentation for lack of a better place, but it's not Plone-specific: http://collective-docs.readthedocs.org/en/latest/zope_secrets/index.html Topics covered include startup, publication, traversal and security. One reason to do this, apart from morbid fascination, is to provide a baseline against which we can consider simplifying some of this stuff. For example, I'd like to consider an (opt-in) simplification of the publisher and traversal, probably based on a stripped-down and modernised repoze.zope2, which does away with some hooks and edge cases, but is much simpler and easier to understand. Some things we could consider chopping are: - Attribute traversal to anything other than methods at the end of the traversal chain (i.e. use __getitem__ traversal only) - Traversal to anything without explicit security declarations - The docstring security check - Maybe __bobo_traverse__ (i.e. just implement __getitem__) and __before_publishing_traverse__ (use a BeforeTraverseEvent instead, and notify this for all traversals, not just over local component sites) - All differences between publication and path traversal This is still somewhat half-baked and obviously would break things and require at least a new major version of Zope, but I think it's worth exploring at least. Martin

4 11

zope-tests - FAILED: 2, OK: 39
by Zope tests summarizer 31 Dec '11

31 Dec '11

This is the summary for test reports received on the zope-tests list between 2011-12-29 00:00:00 UTC and 2011-12-30 00:00:00 UTC: See the footnotes for test reports of unsuccessful builds. An up-to date view of the builders is also available in our buildbot documentation: http://docs.zope.org/zopetoolkit/process/buildbots.html#the-nightly-builds Reports received ---------------- Bluebream / Python2.5.5 64bit linux Bluebream / Python2.6.7 64bit linux Bluebream / Python2.7.2 64bit linux ZTK 1.0 / Python2.4.6 Linux 64bit ZTK 1.0 / Python2.5.5 Linux 64bit ZTK 1.0 / Python2.6.7 Linux 64bit ZTK 1.0dev / Python2.4.6 Linux 64bit ZTK 1.0dev / Python2.5.5 Linux 64bit ZTK 1.0dev / Python2.6.7 Linux 64bit ZTK 1.1 / Python2.5.5 Linux 64bit ZTK 1.1 / Python2.6.7 Linux 64bit ZTK 1.1 / Python2.7.2 Linux 64bit ZTK 1.1dev / Python2.5.5 Linux 64bit ZTK 1.1dev / Python2.6.7 Linux 64bit [1] ZTK 1.1dev / Python2.7.2 Linux 64bit Zope 3.4 KGS / Python2.4.6 64bit linux Zope 3.4 KGS / Python2.5.5 64bit linux Zope 3.4 Known Good Set / py2.4-32bit-linux Zope 3.4 Known Good Set / py2.4-64bit-linux Zope 3.4 Known Good Set / py2.5-32bit-linux Zope 3.4 Known Good Set / py2.5-64bit-linux Zope Buildbot / zope2.12-py2.6 slave-ubuntu64 Zope Buildbot / zope2.13-py2.6 slave-ubuntu64 Zope Buildbot / zope2.13-py2.7 slave-ubuntu64 Zope Buildbot / zope2.14-py2.6 slave-ubuntu64 Zope Buildbot / zope2.14-py2.7 slave-ubuntu64 Zope Buildbot / zopetoolkit-1.0-py2.4 slave-ubuntu64 Zope Buildbot / zopetoolkit-1.0-py2.5 slave-ubuntu64 Zope Buildbot / zopetoolkit-1.0-py2.6 slave-ubuntu64 Zope Buildbot / zopetoolkit-1.1-py2.5 slave-ubuntu64 Zope Buildbot / zopetoolkit-1.1-py2.6 slave-ubuntu64 [2] Zope Buildbot / zopetoolkit-py2.5 slave-ubuntu64 Zope Buildbot / zopetoolkit-py2.6 slave-ubuntu64 Zope-2.10 Python-2.4.6 : Linux Zope-2.11 Python-2.4.6 : Linux Zope-2.12 Python-2.6.6 : Linux Zope-2.12-alltests Python-2.6.6 : Linux Zope-2.13 Python-2.6.6 : Linux Zope-2.13-alltests Python-2.6.6 : Linux Zope-trunk Python-2.6.6 : Linux Zope-trunk-alltests Python-2.6.6 : Linux Non-OK results -------------- [1] FAILED ZTK 1.1dev / Python2.7.2 Linux 64bit https://mail.zope.org/pipermail/zope-tests/2011-December/055014.html [2] FAILED Zope Buildbot / zopetoolkit-py2.5 slave-ubuntu64 https://mail.zope.org/pipermail/zope-tests/2011-December/055032.html

2 1

zope-tests - FAILED: 2, OK: 39
by Zope tests summarizer 30 Dec '11

30 Dec '11

This is the summary for test reports received on the zope-tests list between 2011-12-28 00:00:00 UTC and 2011-12-29 00:00:00 UTC: See the footnotes for test reports of unsuccessful builds. An up-to date view of the builders is also available in our buildbot documentation: http://docs.zope.org/zopetoolkit/process/buildbots.html#the-nightly-builds Reports received ---------------- Bluebream / Python2.5.5 64bit linux Bluebream / Python2.6.7 64bit linux Bluebream / Python2.7.2 64bit linux ZTK 1.0 / Python2.4.6 Linux 64bit ZTK 1.0 / Python2.5.5 Linux 64bit ZTK 1.0 / Python2.6.7 Linux 64bit ZTK 1.0dev / Python2.4.6 Linux 64bit ZTK 1.0dev / Python2.5.5 Linux 64bit ZTK 1.0dev / Python2.6.7 Linux 64bit ZTK 1.1 / Python2.5.5 Linux 64bit ZTK 1.1 / Python2.6.7 Linux 64bit ZTK 1.1 / Python2.7.2 Linux 64bit ZTK 1.1dev / Python2.5.5 Linux 64bit ZTK 1.1dev / Python2.6.7 Linux 64bit [1] ZTK 1.1dev / Python2.7.2 Linux 64bit Zope 3.4 KGS / Python2.4.6 64bit linux Zope 3.4 KGS / Python2.5.5 64bit linux Zope 3.4 Known Good Set / py2.4-32bit-linux Zope 3.4 Known Good Set / py2.4-64bit-linux Zope 3.4 Known Good Set / py2.5-32bit-linux Zope 3.4 Known Good Set / py2.5-64bit-linux Zope Buildbot / zope2.12-py2.6 slave-ubuntu64 Zope Buildbot / zope2.13-py2.6 slave-ubuntu64 Zope Buildbot / zope2.13-py2.7 slave-ubuntu64 Zope Buildbot / zope2.14-py2.6 slave-ubuntu64 Zope Buildbot / zope2.14-py2.7 slave-ubuntu64 Zope Buildbot / zopetoolkit-1.0-py2.4 slave-ubuntu64 Zope Buildbot / zopetoolkit-1.0-py2.5 slave-ubuntu64 Zope Buildbot / zopetoolkit-1.0-py2.6 slave-ubuntu64 Zope Buildbot / zopetoolkit-1.1-py2.5 slave-ubuntu64 Zope Buildbot / zopetoolkit-1.1-py2.6 slave-ubuntu64 [2] Zope Buildbot / zopetoolkit-py2.5 slave-ubuntu64 Zope Buildbot / zopetoolkit-py2.6 slave-ubuntu64 Zope-2.10 Python-2.4.6 : Linux Zope-2.11 Python-2.4.6 : Linux Zope-2.12 Python-2.6.6 : Linux Zope-2.12-alltests Python-2.6.6 : Linux Zope-2.13 Python-2.6.6 : Linux Zope-2.13-alltests Python-2.6.6 : Linux Zope-trunk Python-2.6.6 : Linux Zope-trunk-alltests Python-2.6.6 : Linux Non-OK results -------------- [1] FAILED ZTK 1.1dev / Python2.7.2 Linux 64bit https://mail.zope.org/pipermail/zope-tests/2011-December/054973.html [2] FAILED Zope Buildbot / zopetoolkit-py2.5 slave-ubuntu64 https://mail.zope.org/pipermail/zope-tests/2011-December/055000.html

1 0

ZopeSecurityPolicy global manipulation
by Martin Aspeli 29 Dec '11

29 Dec '11

Hi, AccessControl.ZopeSecurityPolicy contains this code: from types import MethodType # AccessControl.Implementation inserts: # ZopeSecurityPolicy, getRoles, rolesForPermissionOn from AccessControl.SimpleObjectPolicies import _noroles rolesForPermissionOn = None # XXX: avoid import loop tuple_or_list = tuple, list def getRoles(container, name, value, default): global rolesForPermissionOn # XXX: avoid import loop if rolesForPermissionOn is None: from PermissionRole import rolesForPermissionOn roles = getattr(value, '__roles__', _noroles) if roles is _noroles: if not name or not isinstance(name, basestring): return default if type(value) is MethodType: container = value.im_self cls = getattr(container, '__class__', None) if cls is None: return default roles = getattr(cls, name+'__roles__', _noroles) if roles is _noroles: return default value = container if roles is None or isinstance(roles, tuple_or_list): return roles rolesForPermissionOn = getattr(roles, 'rolesForPermissionOn', None) if rolesForPermissionOn is not None: roles = rolesForPermissionOn(value) return roles Look carefully at how ``rolesForPermissionOn`` is used both at the top, to lazily set a global to avoid an import loop, and the bottom, as an attribute of the ``roles`` object. I'm pretty sure this is wrong™ on many levels, but most importantly, it seems the global is being overwritten each time execution gets down to that last block. I know this module gets munged by Implementation, but I'm pretty sure ImplPython doesn't define getRoles() at least, and I'm not even sure the C implementation does either. To prove it to myself, I made a frivolous equivalent that used 'datetime.date' as the importable. It's a bit ugly, but you get the idea: >>> date = None >>> class C(object): ... def __init__(self, d): ... self.date = d ... >>> c1 = C(lambda: 'x') >>> c2 = C(lambda: 'y') >>> def get(c): ... global date ... if date is None: ... from datetime import date ... date = getattr(c, 'date', None) ... if date is not None: ... print date() ... >>> date is None True >>> get(c1) x >>> date <function <lambda> at 0x10dac8140> >>> get(c2) y >>> date <function <lambda> at 0x10dac8cf8> >>> Surely, this is all evil volatile? Maybe the global bit just needs to go away? It doesn't seem to be used in that function, and I'm pretty sure the implementation ends up overwriting the global anyway. Cheers, Martin

1 0

PAS and AccessControl bug?
by Martin Aspeli 29 Dec '11

29 Dec '11

Hi, I found this code in PAS, which is mostly lifted from AccessControl.userfolder: def _getObjectContext( self, v, request ): """ request -> ( a, c, n, v ) o 'a 'is the object the object was accessed through o 'c 'is the physical container of the object o 'n 'is the name used to access the object o 'v' is the object (value) we're validating access to o XXX: Lifted from AccessControl.User.BasicUserFolder._getobcontext """ if len( request.steps ) == 0: # someone deleted root index_html request[ 'RESPONSE' ].notFoundError( 'no default view (root default view was probably deleted)' ) root = request[ 'PARENTS' ][ -1 ] request_container = aq_parent( root ) n = request.steps[ -1 ] # default to accessed and container as v.aq_parent a = c = request[ 'PARENTS' ][ 0 ] # try to find actual container inner = aq_inner( v ) innerparent = aq_parent( inner ) if innerparent is not None: # this is not a method, we needn't treat it specially c = innerparent elif hasattr(v, 'im_self'): # this is a method, we need to treat it specially c = v.im_self c = aq_inner( v ) # if pub's aq_parent or container is the request container, it # means pub was accessed from the root if a is request_container: a = root if c is request_container: c = root return a, c, n, v Look at this bit again: elif hasattr(v, 'im_self'): # this is a method, we need to treat it specially c = v.im_self c = aq_inner( v ) In AccessControl, it's similar: elif hasattr(v, 'im_self'): # this is a method, we need to treat it specially c = v.im_self c = getattr(v, 'aq_inner', v) Surely, this isn't right? What is the correct thing to do here? Martin

1 0

zope-tests - FAILED: 2, OK: 39
by Zope tests summarizer 29 Dec '11

29 Dec '11

This is the summary for test reports received on the zope-tests list between 2011-12-27 00:00:00 UTC and 2011-12-28 00:00:00 UTC: See the footnotes for test reports of unsuccessful builds. An up-to date view of the builders is also available in our buildbot documentation: http://docs.zope.org/zopetoolkit/process/buildbots.html#the-nightly-builds Reports received ---------------- Bluebream / Python2.5.5 64bit linux Bluebream / Python2.6.7 64bit linux Bluebream / Python2.7.2 64bit linux ZTK 1.0 / Python2.4.6 Linux 64bit ZTK 1.0 / Python2.5.5 Linux 64bit ZTK 1.0 / Python2.6.7 Linux 64bit ZTK 1.0dev / Python2.4.6 Linux 64bit ZTK 1.0dev / Python2.5.5 Linux 64bit ZTK 1.0dev / Python2.6.7 Linux 64bit ZTK 1.1 / Python2.5.5 Linux 64bit ZTK 1.1 / Python2.6.7 Linux 64bit ZTK 1.1 / Python2.7.2 Linux 64bit ZTK 1.1dev / Python2.5.5 Linux 64bit ZTK 1.1dev / Python2.6.7 Linux 64bit [1] ZTK 1.1dev / Python2.7.2 Linux 64bit Zope 3.4 KGS / Python2.4.6 64bit linux Zope 3.4 KGS / Python2.5.5 64bit linux Zope 3.4 Known Good Set / py2.4-32bit-linux Zope 3.4 Known Good Set / py2.4-64bit-linux Zope 3.4 Known Good Set / py2.5-32bit-linux Zope 3.4 Known Good Set / py2.5-64bit-linux Zope Buildbot / zope2.12-py2.6 slave-ubuntu64 Zope Buildbot / zope2.13-py2.6 slave-ubuntu64 Zope Buildbot / zope2.13-py2.7 slave-ubuntu64 Zope Buildbot / zope2.14-py2.6 slave-ubuntu64 Zope Buildbot / zope2.14-py2.7 slave-ubuntu64 Zope Buildbot / zopetoolkit-1.0-py2.4 slave-ubuntu64 Zope Buildbot / zopetoolkit-1.0-py2.5 slave-ubuntu64 Zope Buildbot / zopetoolkit-1.0-py2.6 slave-ubuntu64 Zope Buildbot / zopetoolkit-1.1-py2.5 slave-ubuntu64 Zope Buildbot / zopetoolkit-1.1-py2.6 slave-ubuntu64 [2] Zope Buildbot / zopetoolkit-py2.5 slave-ubuntu64 Zope Buildbot / zopetoolkit-py2.6 slave-ubuntu64 Zope-2.10 Python-2.4.6 : Linux Zope-2.11 Python-2.4.6 : Linux Zope-2.12 Python-2.6.6 : Linux Zope-2.12-alltests Python-2.6.6 : Linux Zope-2.13 Python-2.6.6 : Linux Zope-2.13-alltests Python-2.6.6 : Linux Zope-trunk Python-2.6.6 : Linux Zope-trunk-alltests Python-2.6.6 : Linux Non-OK results -------------- [1] FAILED ZTK 1.1dev / Python2.7.2 Linux 64bit https://mail.zope.org/pipermail/zope-tests/2011-December/054928.html [2] FAILED Zope Buildbot / zopetoolkit-py2.5 slave-ubuntu64 https://mail.zope.org/pipermail/zope-tests/2011-December/054950.html

2 1