October 2003 - Zope-Dev

relations in objects
by Jason Corbett 28 Dec '03

28 Dec '03

Hello Zope developers, I'm a newbie (sorry most people have this curse at one point or another). I have read the zope book (or at least most of it), and the zope developer manual. I can't quite figure out how to have relationships between instances of classes in the database. Let me first ask if I'm using zope for the correct thing. I see that a lot of zope products deal with content management, while mine doesn't really. Basically I want to do what I would normally do with mysql and php, but use zope instead. So I need a way to reference in some intelligent way between objects in zope, and I need a way to query for a list of them. I read about ZCatalogs, etc and how you can use that for querying and indexing. That isn't my first priority, so I'll worry about that a little later. Basically I have an instance of a class 'A'. As part of it I need to reference either a single or in some cases multiple instances of class 'B'. Basically I'll have a mix of one-to-one relations and one-to-many. When I'm coding my python class (I'm going to do a product if you can't tell), how would I get a list of the 'B' instances that relate to this object, and how would I store that? The zope developer manual was pretty good for writing classes, and making views on it, but I need to be able to call methods on those instances and return lists of them. Sorry for the long email, I just don't want to leave details out. If you think I'm barking up the wrong tree, and shouldn't be using zope for this, don't be afraid to let me know. Although I will be pretty disappointed, I think zope is exactly what I've been looking for, I can't find anything like it in any other language. Jason Corbett __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com

7 10

New-style ExtensionClass
by Jim Fulton 02 Dec '03

02 Dec '03

I've started work on a new-style ExtensionClass. This is a port of ExtensionClass to new-style classes. This will provide a number of advantages: - Use of new-style class features (e.g. descriptors) in Zope objects. - Support for cyclic garbage collection. - Ability to use new-style classes as base classes of Zope objects. - Use of a version of ZODB that supports non-ExtensionClass classes. - Pave the way for sharing code between Zope 2 and Zope 3. I hope I can merge this into the Zope 2 head in a week or two. This is a rather deep change and it is likely to cause some instability on the CVS head for a while. I'm doing this now, rather than later, to give us plenty of time to find and fix problems before a Zope 2.8 release. Speaking of Zope 2.8, Jeremy Hylton has suggested that, perhaps, Zope 2.8 should be a release that provides *only*: - New-style ExtensionClass, and - ZODB 3.3, featuring multi-version concurrency control, plus any features that have been added to the head since the Zope 2.7 branch was created. This idea is pretty appealing to me. I wonder what others think of it. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

8 11

New-style ExtensionClasses (Zope 2.8) -- MRO issue
by Jim Fulton 29 Nov '03

29 Nov '03

I've been working on a "new-style" ExtensionClass (nsEC) project to reimplement ExtensionClass as a new-style meta class. This will allow ExtensionClasses (i.e. most Zope 2 classes) to be able to use features of new-style Python classes, including: - New protocols, - Descriptors, - Garbage Collection, ... I think I've got this largely working, but have encountered a significant issue that I'd like some input on. New-style classes and classic classes use different algorithms for looking up inherited attributes (e.g. methods). Classic classes use a left-to-right depth-first algorithm. Python 2.3 uses a "C3 Method-Resolution Order" algorithm. I won't try to explain how this algorithm works. If you are curious, see: http://www.python.org/2.3/mro.html This new algorithm has a couple of noteworthy properties: - It yields different lookup results than the left-to-right depth-first algorithm used for classic classes. - It is sometimes impossible to compute a "method resolution order" for a given class and base classes. >>> class A(object): ... pass >>> class B(A): ... pass >>> class C(A, B): ... pass Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: Cannot create a consistent method resolution order (MRO) for bases B, A The original ExtensionClass used the left-to-right depth-first algorithm used for classic classes. Currently, nsEC inherits the new method lookup algorithm from type (the standard Python 2.3 metaclass used for new-style classes). This has run into significant difficulties with the Zope 2 classes, which have a highly complex inheritence graph. An initial analysis showed that for more than half of the classes in Zope 2, it was impossible to compute a method-resolution order using the new algorithm. After about a day of analysis (and development of a tool to aid in the analysis) I've been able to adjust the Zope classes so that they will work with the new algorithm. I also analyzed the CMF head. To use the new algorithm with the CMF, it was necessary to make some changes to the CMF and to Zope. In one case, it was necessary to copy some methods around. An interesting result of the analysis is that it identified several methods that were being inherited incorrectly. The new inheritence algorithm yeilded better results for these cases, once the classes were modified enough to use the algorithm at all. It is very likely that, if we use the new algorithm for nsEC, many products will not work anymore (meaning with Zope 2.8). Modifying the products to work will probably be a non-trivial undertaking. We have three alternatives: 1. Use the old method-lookup algorithm for nsEC. Pro: backward compatible Con: may produce odd results if a new-style class with a complex arrangement of base classes is mixed with an ExtensionClass. 2. Use the new-style method-lookup algithm Pro: Most consistent with new-style classes. Cons: Likely that many products will be broken and require change to work with nsEC/Zope 2.8. Analysis of the changes needed to fix method-resolution-order problems is extremely complex. 3. Use a hybrid schema. I'll call this the "encapsulated base" scheme. Consider: >>> class C(X, Y, Z): ... def foo(self): ... ... ... When we look up an attribute, we do the following: - Look in C's dictionary first. All three algorithms agree on this. :) - Look up the attribute in X. We don't care how we het the attribute from X. If X is a new-style-class, we use the new algithm. If X is a classic class, we use left-to-right depth-first. If X is an nsEC, use the "encapsulated base" algithm. If we don't find the attribute in X, look in Y and then in Z, using the same approach. This algithm will produce backward compatible results, providing the equivalent of left-to-right depth-first for nsECs and classic classes. (Note for the more advanced reader: we'll actually do something less abstract. We'll use a simple algorthm to merge the __mro__ of the base classes, computing an __mro__ for classes classes using the left-to-right depth-first algorithm. We'll basically lay the mros end-to-end left-to-right and remove repeats, keeping the first occurence of each class.) I find this approach appealing for reasons of encapsulation. When reasoning about C, I can reason from known behavior of X, Y, and Z, without considering their respective inherietnce hierarchies, which I consider to be an implementation detail. Pros: backward compatible Less likely than option 1 to produce odd results if a new-style class with a complex arrangement of base classes is mixed with an ExtensionClass. Con: may produce odd results if a new-style class with a complex arrangement of base classes is mixed with an ExtensionClass. Thoughts? I am worried enough about breaking products that I'm inclined to go with option 3. Does anybody think we ought to use the new algorithm (option 2)? Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

11 28

Question about 'The Zope Developer's Guide'
by Baby Zope 08 Nov '03

08 Nov '03

Hi ! I have tried to find 'The Zope Developer's Guide' without lucky. Could you give me a location to download it? Thanks. _________________________________________________________________ Accede al romance online. Busca y encuentra a tu media naranja entre los perfiles que te interesan. http://match.msn.es/

2 1

debugging memory leaks
by Seb Bacon 06 Nov '03

06 Nov '03

I've got a fairly major memory leak in my application. I've followed the thread from August including Shane's suggestions about using a debug build of Python to inspect object references and the rest [1] I know from the refcounts in Zope that items of class Foo are definitely leaking, yet when I do a sys.getobjects(0, Foo) I get nothing back. Navigating all 100000-ish references one by one seems a bit daunting. So I'm not sure where to go from here. Shane, you mentioned you had put together some useful functions for exploring this debug information - could you share them? Or were you just refering to the remote console you supplied earlier? Also, what kinds of Python code can cause memory leaks? I'm not really sure what I should be looking for. The only thing I can think of is hanging file descriptors, as circular references should be picked up by the gc anyway. Seb [1] http://mail.zope.org/pipermail/zope-dev/2003-August/020358.html

3 6

Zope doesn't know enough mime types.
by Romain Slootmaekers 04 Nov '03

04 Nov '03

Yo, Zope doesn't know enough mime types, and there's no easy way to add them add runtime since you removed the add_type(...) method from mimetypes.py mime types that spring to mind are: .ra .ram .rm : audio/x-pn-realaudio .rpm : audio/x-pn-realaudio-plugin .jad : text/vnd.sun.j2me.app-descriptor .jar : application/java-archive I'm sure there must be quite a few others. Maybe someone can add the lot of'em. This also is vaguely related to: http://zope.org/Collectors/Zope/969 TIA, Romain Slootmaekers

8 18

Bugday Reloaded: The Sequel
by Brian Lloyd 03 Nov '03

03 Nov '03

Hi all - I'd like to try to have another bug day next Tuesday (Nov 3) to make up for the one that was cut short last week, in preparation for a beta 2 release of Zope 2.7. I know its short notice, but every little bit helps. If you can make it even only for an hour or two it will be much appreciated. thanks! Brian Lloyd brian(a)zope.com V.P. Engineering 540.361.1716 Zope Corporation http://www.zope.com

2 2

Bare except and ConflictError in manage_beforeDelete
by Florent Guillaume 02 Nov '03

02 Nov '03

I have code that needs to maintain certain invariants when objects are deleted, these invariants are maintained in manage_beforeDelete. Unfortunately manage_beforeDelete has this code: try: if hasattr(aq_base(object), 'manage_beforeDelete'): object.manage_beforeDelete(item, container) except BeforeDeleteException, ob: raise except: LOG('Zope',ERROR,'manage_beforeDelete() threw', error=sys.exc_info()) pass Which means that ConflictErrors in my invariant code get swallowed, and my invariants are broken. This kills me. I want to add an additionnal: except ConflictError: raise Any objections ? Florent -- Florent Guillaume, Nuxeo (Paris, France) +33 1 40 33 79 87 http://nuxeo.com mailto:fg@nuxeo.com

4 4

Zope credits
by Jeremy Hylton 31 Oct '03

31 Oct '03

Dean Goodmanson pointed out that the Zope credits file is a little out of date: http://www.pycs.net/sqr/categories/slicesOfPy/2003/10/30.html#a184 Wouldn't it be nice to get it in better shape for the 2.7 release? It seems like a task that is easy to do in parallel. If credits are too hard, how about an ACKS file like Python's? Jeremy

1 0

Member tools refactoring (Was: Better MemberData search for CMF)
by Lennart Regebro 31 Oct '03

31 Oct '03

Hi all! I'm gonna try to verbalize my thoughts on this subject, but they are still quite fuzzy, so bear with me if I sound confused. :) And sorry for the length of tha mail... Today there are many parts involved in the Member Data Waltz (or is it a two-step). There is the user folder, of course, there is the portal_membership tool and there is the portal_memberdata tool. So what you do is that you ask the portal_membership tool for the information, but it takes the user object from the user folder (which already keeps contains some user data) and then asks portal_memberdata to wrap the user with it's data. Then there is also the portal_registration tool, who also does most of it's things by asking portal_membership. :) Missing from this is both the possibility to do transforms of data, and also the possibility to have several user directories, like for example having most users in an LDAP directory, but some users (like managers) local. To solve this, in CPS there is a portal_metadirectories tool involved (I don't know if there are other alternatives to this for non CPS use), so then we have five different "tools", three of which may store some parts of the actual user data, and all which seem to overlap in functionality in different ways. My suggestion of how to do all this is refactor this into the following pieces: 1. A user folder 2. A user directory tool 3. A membership tool 1. The user folder stays as it is. It will functionally overlap with the user directory a lot, but that is to allow third-party userfolders to still work. I would actually not mind seeing all of the three parts above merged, but I think that should be a Zope 3 thing, in that case. The search API here need only return a list of userIds, btw. If you want the data, ask the membership tool. 2. The user directory tool would basically be a merge of the memberdata tool and CPS's memberdirectory tool. It would contain a set of directories. One could be internal, others could be external. For example one LDAP directory, and one Zope directory. These directories should all implement advanced search functionality. The main directory tool would perform a search by calling the searching on all directories. The tool need to be able to perform transforms. One source may for example store a users fullname in the fields "Name" and "Firstname" another may store it as "Name" and "Family name" and a third just as "Fullname". The tools would need to be able to create a "fullname" property out of these properties. The data may need to be merged for all the users, since you may want to store user data in the Zope directory also for LDAP users. Or maybe this is overkill, I'm not sure. 3. A membership tool, that merges todays membership tool with the registration tool. Again, I think this really should belong in acl_users, but that is not feasible for Zope 2. OK, flame away! //Lennart

2 1