At 05:10 PM 11/5/99 -0800, Jonothan Farr wrote:

...

Does anyone know whether Acquisition works in a way which is compatible

with

...

__getattr__?

I'm really interested in the answer to this, too. I had no luck getting __getattr__ to work with acquisition when developing the local file system product. What are the rules? Is this even possible?

When acquisition wrappers go looking for attributes, they will call your __getattr__, but it is with an unwrapped self. This means that acquisition can use __getattr__ routines, but __getattr__ routines cannot make use of the object's acquired attributes. There is *no* workaround, short of rewriting the Acquisition ExtensionClass and creating an alternative application framework to Zope. (I actually did this myself, to solve this very problem of needing "wrapped __getattr__", but of course the resulting framework can't use Zope products. Oh well.)

Michel Pelletier wrote:

That's the theory. Way back in the day, a few months ago, I tried to write a BTreeObjectManager that stored it's children in a dictionary (a BTree, but it acts just like a dict and it's faster and much more space efficient when working with ZODB). I got pretty far along, and it seemed to work ok, but there were bugs I never had time to fix. It was just an experiment. if I can dig up the code, I'll post it.

-Michel

Code is attached. Code is as is, buggy, and purely experimental. Consists of two file, BTreeObjectManager.py and BTreeFolder.py. Both go in OFS (but it would probably be smart to make a seperate Product). -Michel ############################################################################## # # Zope Public License (ZPL) Version 1.0 # ------------------------------------- # # Copyright (c) Digital Creations. All rights reserved. # # This license has been certified as Open Source(tm). # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are # met: # # 1. Redistributions in source code must retain the above copyright # notice, this list of conditions, and the following disclaimer. # # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions, and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # 3. Digital Creations requests that attribution be given to Zope # in any manner possible. Zope includes a "Powered by Zope" # button that is installed by default. While it is not a license # violation to remove this button, it is requested that the # attribution remain. A significant investment has been put # into Zope, and this effort will continue if the Zope community # continues to grow. This is one way to assure that growth. # # 4. All advertising materials and documentation mentioning # features derived from or use of this software must display # the following acknowledgement: # # "This product includes software developed by Digital Creations # for use in the Z Object Publishing Environment # (http://www.zope.org/)." # # In the event that the product being advertised includes an # intact Zope distribution (with copyright and license included) # then this clause is waived. # # 5. Names associated with Zope or Digital Creations must not be used to # endorse or promote products derived from this software without # prior written permission from Digital Creations. # # 6. Modified redistributions of any form whatsoever must retain # the following acknowledgment: # # "This product includes software developed by Digital Creations # for use in the Z Object Publishing Environment # (http://www.zope.org/)." # # Intact (re-)distributions of any official Zope release do not # require an external acknowledgement. # # 7. Modifications are encouraged but must be packaged separately as # patches to official Zope releases. Distributions that do not # clearly separate the patches from the original work must be clearly # labeled as unofficial distributions. Modifications which do not # carry the name Zope may be packaged in any form, as long as they # conform to all of the clauses above. # # # Disclaimer # # THIS SOFTWARE IS PROVIDED BY DIGITAL CREATIONS ``AS IS'' AND ANY # EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR # PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL DIGITAL CREATIONS OR ITS # CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF # USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND # ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # # This software consists of contributions made by Digital Creations and # many individuals on behalf of Digital Creations. Specific # attributions are listed in the accompanying credits file. # ############################################################################## __doc__="""Object Manager $Id: ObjectManager.py,v 1.80 1999/07/22 19:19:22 jim Exp $""" __version__='$Revision: 1.80 $'[11:-2] from BTree import BTree import ts_regex, Globals from ObjectManager import ObjectManager bad_id=ts_regex.compile('[^a-zA-Z0-9-_~\,\. ]').match #TS _marker=[] class BTreeObjectManager(ObjectManager): """BTree object manager This class provides core behavior for collections of heterogeneous objects. It's API is identical to that of the standard Zope ObjectManager. Instead of storing it's objects sequentially, the BTreeObjectManager stores its objects in very efficient BTree data structures written in C. """ meta_type ='BTree Object Manager' _objects = () _real_objects = BTree() def _checkId(self, id, allow_dup=0): # If allow_dup is false, an error will be raised if an object # with the given id already exists. If allow_dup is true, # only check that the id string contains no illegal chars. if not id: raise 'Bad Request', 'No id was specified' if bad_id(id) != -1: raise 'Bad Request', ( 'The id %s contains characters illegal in URLs.' % id) if id[0]=='_': raise 'Bad Request', ( 'The id %s is invalid - it begins with an underscore.' % id) if not allow_dup: if hasattr(self, 'aq_base'): self=self.aq_base if self._real_objects.has_key(id): raise 'Bad Request', ( 'The id %s is invalid - it is already in use.' % id) def _setOb(self, id, object): self._real_objects[id] = object def _delOb(self, id): del self._real_objects[id] def _getOb(self, id, default=_marker): if hasattr(self, 'aq_base'): base=self.aq_base else: base=self if not self._real_objects.has_key(id): if default is _marker: raise AttributeError, id return default return self._real_objects[id].__of__(self) Globals.default__class_init__(BTreeObjectManager) ############################################################################## # # Zope Public License (ZPL) Version 1.0 # ------------------------------------- # # Copyright (c) Digital Creations. All rights reserved. # # This license has been certified as Open Source(tm). # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are # met: # # 1. Redistributions in source code must retain the above copyright # notice, this list of conditions, and the following disclaimer. # # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions, and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # 3. Digital Creations requests that attribution be given to Zope # in any manner possible. Zope includes a "Powered by Zope" # button that is installed by default. While it is not a license # violation to remove this button, it is requested that the # attribution remain. A significant investment has been put # into Zope, and this effort will continue if the Zope community # continues to grow. This is one way to assure that growth. # # 4. All advertising materials and documentation mentioning # features derived from or use of this software must display # the following acknowledgement: # # "This product includes software developed by Digital Creations # for use in the Z Object Publishing Environment # (http://www.zope.org/)." # # In the event that the product being advertised includes an # intact Zope distribution (with copyright and license included) # then this clause is waived. # # 5. Names associated with Zope or Digital Creations must not be used to # endorse or promote products derived from this software without # prior written permission from Digital Creations. # # 6. Modified redistributions of any form whatsoever must retain # the following acknowledgment: # # "This product includes software developed by Digital Creations # for use in the Z Object Publishing Environment # (http://www.zope.org/)." # # Intact (re-)distributions of any official Zope release do not # require an external acknowledgement. # # 7. Modifications are encouraged but must be packaged separately as # patches to official Zope releases. Distributions that do not # clearly separate the patches from the original work must be clearly # labeled as unofficial distributions. Modifications which do not # carry the name Zope may be packaged in any form, as long as they # conform to all of the clauses above. # # # Disclaimer # # THIS SOFTWARE IS PROVIDED BY DIGITAL CREATIONS ``AS IS'' AND ANY # EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR # PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL DIGITAL CREATIONS OR ITS # CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF # USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND # ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # # # This software consists of contributions made by Digital Creations and # many individuals on behalf of Digital Creations. Specific # attributions are listed in the accompanying credits file. # ############################################################################## """Folder object Folders are the basic container objects and are analogous to directories. $Id: Folder.py,v 1.82 1999/07/05 14:44:49 jim Exp $""" __version__='$Revision: 1.82 $'[11:-2] import Globals, SimpleItem from BTreeObjectManager import BTreeObjectManager from PropertyManager import PropertyManager from AccessControl.Role import RoleManager from webdav.Collection import Collection from FindSupport import FindSupport from Globals import HTMLFile manage_addBTreeFolderForm=HTMLFile('btreeFolderAdd', globals()) def manage_addBTreeFolder(self, id, title='', createPublic=0, createUserF=0, REQUEST=None): """Add a new BTreeFolder object with id *id*. If the 'createPublic' and 'createUserF' parameters are set to any true value, an 'index_html' and a 'UserBTreeFolder' objects are created respectively in the new folder. """ ob=BTreeFolder() ob.id=id ob.title=title self._setObject(id, ob) try: user=REQUEST['AUTHENTICATED_USER'] except: user=None if createUserF: if (user is not None) and not ( user.has_permission('Add User Folders', self)): raise 'Unauthorized', ( 'You are not authorized to add User Folders.' ) ob.manage_addUserFolder() if createPublic: if (user is not None) and not ( user.has_permission('Add Documents, Images, and Files', self)): raise 'Unauthorized', ( 'You are not authorized to add DTML Documents.' ) ob.manage_addDTMLDocument(id='index_html', title='') if REQUEST is not None: return self.manage_main(self, REQUEST, update_menu=1) class BTreeFolder(BTreeObjectManager, PropertyManager, RoleManager, Collection, SimpleItem.Item, FindSupport): """ The basic container object in Principia. BTreeFolders can hold almost all other Principia objects. """ meta_type='BTree Folder' _properties=({'id':'title', 'type': 'string'},) manage_options=( {'label':'Contents', 'action':'manage_main'}, {'label':'View', 'action':'index_html'}, {'label':'Properties', 'action':'manage_propertiesForm'}, {'label':'Import/Export', 'action':'manage_importExportForm'}, {'label':'Security', 'action':'manage_access'}, {'label':'Undo', 'action':'manage_UndoForm'}, {'label':'Find', 'action':'manage_findFrame', 'target':'manage_main'}, ) __ac_permissions__=() Globals.default__class_init__(BTreeFolder)

Ty Sarna wrote:

In article <3826F005.7429DD42@digicool.com>, Michel Pelletier <michel@digicool.com> wrote:

...

Consists of two file, BTreeObjectManager.py and BTreeFolder.py. Both go in OFS (but it would probably be smart to make a seperate Product).

BTW, I wrote something like this a month back or so called ZRack, which is similar. It was based onm BTree at first, but I later changed it to use Phillip Eby's simpletree, which has better write performance (doesn't always have to update one page per level of the tree). It also will assign id's for you, if you want.

BTW, we plan of fixing the update-one-per-level bug, which Phillip pointed out to us. Can't make any promises on time...

At any rate, one problem I ran into is that the ZClass support doesn't interract well with objectmanager's that don't support __getattr__. Adding a ZClass instance wants to add the instance, and then immediately getattr it back out.

Hmm.. this sounds like more of a bug in ZClasses to me than OMs, but I see your point below, it's really about wanting to override __getattr__

It was easy to hack around this, but it sure would be nice if the tree-ObjectManager *could* support getattr. The inability to override __getattr_ on Persistent subclasses keeps biting me (and others, from what I see on the lists) again and again. It would be *really* nice if this could be fixed. I'd even be happy with something like "You can't override __getattr_, but if you define __foo_getattr__, persistent will try it before returning AttributeError".

I suspect jim would frown on another hook, we're trying to avoid as many new hooks as possible, especialy ones that may have a per hit or per traversal penalty. Perhaps I haven't thought about your idea enough though. -Michel

At 09:34 PM 11/11/99 GMT, Ty Sarna wrote:

In article <382B335F.3160393A@digicool.com>, Michel Pelletier <michel@digicool.com> wrote:

...

I suspect jim would frown on another hook, we're trying to avoid as many new hooks as possible, especialy ones that may have a per hit or per traversal penalty. Perhaps I haven't thought about your idea enough though.

Right. I'm just saying that if there does have to be a gross hack, I could accept that. It would be nice if it were simply possible to override __getattr__ in the usual manner. Or in an unusual manner, for that matter :-). So long as there's some way, I'm happy :-)

Actually, here's an unusual manner in which you can do what you want (mostly). :) If you want a Persistent object that does __getattr__, all you need to do is redefine its __of__ method thusly: def __of__(self,context): me = Acquisiton.ImplicitAcquisitonWrapper(self,context) # normal self return Acquisiton.ImplicitAcquisitonWrapper(GetAttrWrapper(me),context) and define GetAttrWrapper thusly: class GetAttrWrapper(Acquisition.Implicit): def __init__(self,other): self._other = other def __getattr__(self,attr): other = self.__dict__['_other'] try: return other[attr] except KeyError: return getattr(other,attr) def __setattr__(self,attr,value): setattr(self.__dict__['_other'],attr,value) This is a very ugly kludge, and there are aspects that may not work 100% correctly. What it does is stick a wrapper around your persistent object which converts __getattr__ to try __getitem__ first. The kludgy part is that methods bound to the inner self will NOT have this getattr, as they will not be bound to the wrapper, and thus during execution of such a method, one must perform self[] calls to access these extended attributes. This could be overcome with more coding to check for PythonMethod objects and rebinding them as Acquisition.c does, but this is too complicated for me to do off the top of my head in this e-mail. :)

At 02:39 PM 11/12/99 +0000, Ty Sarna wrote:

In article <3.0.5.32.19991111184711.0132deb0@telecommunity.com>, Phillip J. Eby <pje@telecommunity.com> wrote:

...

Actually, here's an unusual manner in which you can do what you want (mostly). :)

[snip __of__'ing a wrapper trick]

...

This is a very ugly kludge, and there are aspects that may not work 100% correctly. What it does is stick a wrapper around your persistent object which converts __getattr__ to try __getitem__ first. The kludgy part is

What worries me about thich technique is that other code may break, assuming that it's dealing with the true object (or an acquisition wrapper, which presumably all Zope code already takes into account ;-)

Not really, since this is the "true object", it just is a bit more complex than usual. :) If another object stores a reference to it, however, they may not get exactly what they intended upon later un-pickling. :(

Announcing the PersistWithGetattr.DynPersist class! (the name really needs some work...) Source code at: http://telecommunity.com/pkg/PersistWithGetattr.c This module provides you with a mixin class that can be used to implement a __getattr__-alternative in Persistent objects. Example code: from ZODB.PersistWithGetattr import DynPersist from Persistence import Persistent from Acquisition import Implicit class foo(DynPersist,Persistent,Implicit): def __get_attr__(self,name): if name[:4]=='foo_' and name[-9:]!='__roles__': return name[4:] raise AttributeError,name Note that your __getattr__ must actually be called __get_attr__, and that DynPersist must appear *before* Persistent in your list of base classes. If you are subclassing from another Zope framework class that is already Persistent, just put DynPersist ahead of it in your base class list. The class is implemented in about 100 lines of C code; basically all it does is insert a hook between the normal persistence machinery and the normal ExtensionClass getattr machinery which will look for a __get_attr__ routine. This hooking only takes place for objects which subclass DynPersist, so there is no performance impact on "standard" persistent objects. You should only subclass from DynPersist if your objects have a __get_attr__ method, since DynPersist will only add useless overhead if you do not have one. Build/install instructions: Just add the following line to the Zope2/lib/python/ZODB/Setup file: PersistWithGetattr PersistWithGetattr.c -I../../Components/ExtensionClass And then rebuild Zope. I wrote this code last night, and have done a good bit of testing since, including Ty and I doing an experimental modification to TinyTables to give them a __get_attr__ which lets you access rows as though they were attributes. (This was just for testing and there are currently no plans to release this as a TinyTable feature.) We were able to make changes to TinyTable instances, load data, modify data, and so on, and the changes persisted in the database, and the __get_attr__ hook worked. Changes, and the instances themselves survived multiple server restarts. Our testing did show that defining a useful "__get_attr__" routine for use in the Zope framework can be tough, however, as Zope itself and the acquisition machinery can ask objects for things that seem surprising at first. Unlike a "normal" Python object, where attribute requests are almost always going to be "reasonable" (i.e., things an object would expect/intend to have available), Acquisition will pipe all sorts of attribute requests into your object if you're not careful. And ZPublisher will ask for things like index_html, __doc__, __roles__, name__doc__, name__roles__, and so on and so forth. Our first few attempts at a working __get_attr__ led to some object breakage because we didn't write a __getattr__ that would handle things properly. Here are a couple of tips (which are illustrated by the example above): * Always validate the attribute name against a known list or naming pattern! If the nature of your need is that it must make up things with 'any' name, then at least check whether the name ends in __doc__ or __roles__ (or is 'index_html', '__bobo_traverse__' or another ZPublisher-special name) to help you decide what to do with it. Note that if you are validating against a known list of names (such as using a database or dictionary of some kind), you do not need to check for special names, since they will (presumably) not be in your list. * Always "raise AttributeError,name" if you do not want Zope to think the requested attribute exists. This is important. Once we forgot to do this, and it broke the TinyTable instance we created, because it wound up with an ID of "None". (__get_attr__ apparently returned None when the code fell "off the end" of the method).