ObjectManager example needed
Dear Zopers In continuing my search for help it seems like ObjectManager is what I am looking for in order to have a large collection of different Python objects in the same folder allowing for many links among them. I will need to be able to select which kinds of items to show and sort them and use them in forms. That should be standard behaviour, though, I would expect. Do you happen to have experience with ObjectManager and/or have a good example that I can study? The application I am developing is for a musician who currently has an Excel sheet with more than 20.000 pieces of old music (1600-1900) where each PieceOfMusic object refers to other objects like Composer, OrchestralSetting, Instrument, Cast, Recording, Writer, etc. Those need to become different kinds of Python objects that the PieceOfMusic object can refer to in order to guarantee consistency. I have designed an object model with all the classes, their attributes and relations in a UML diagram, but now I need to convert the Excel sheet into Python objects in Zope and make them accessible through ZPT pages and Python methods based on this UML model. I assume that I need a class MusicLibrary that is a direct subclass of ObjectManager and a variety of classes that represent my application classes and are all subclasses of ObjectManagerItem. Do you agree with this? In the help about ObjectManager and ObjectManagerItem I miss a method like manage_addObject. There are only manage_self, manage_addProduct and manage_addFolder. But which method does the trick of adding Python objects to my product's folder? Any help is appreciated best wishes Andre Meyer
I want to create a site which requires Monkeys to login. Login is done via a form. When a Monkey first creates an account she enters her "monkey_name" and "password" and clicks "submit" Now a script is called which creates a file with Id "monkey_name" containing "password". This bit works fine, no problem. The problem is when the Monkey comes back to log into to their account. They must type in their monkey_name and password. Now, to check if the password is right, I have to open the file named "monkey_name" and check if its content is the same as the password which the monkey typed. This is the script I wrote: ('monkey_name' and 'password' are variables which come from a form) existing_monkeys=container.objectIds('File') no_monkeys=len(existing_monkeys) i=0 for i in range(no_monkeys): if (monkey_name == existing_monkeys[i]): if (container.monkey_name == password): return "Your password is correct." else: return "Your password is incorrect." else: return "there is no monkey by that name." The problem is: 'container.monkey_name' If the script is run with monkey_name = Heidi. What we want is the file: container.Heidi But what Zope looks for is: container.monkey_name, which of course doesn't exist. I checked this by creating a file named Heidi and then a script named cat_file, which contains a single line: return container.Heidi This returned the contents of the file Heidi without any complaints. I then changed cat_file so that it took a parameter file_name return container.file_name If I then call this file cat_file(Heidi) it doesn't work :-( What should I do?
alice wrote at 2003-11-23 17:25 +0700:
... if (monkey_name == existing_monkeys[i]): if (container.monkey_name == password): ... The problem is: 'container.monkey_name'
If the script is run with monkey_name = Heidi. What we want is the file: container.Heidi
Use "getattr(container,monkey_name)", maybe with an optional (third) default value. -- Dieter
Andre, I think you would be much better off using a relational database management system (RDBMS) like MySQL to contain the information abut the music and then use Zopes interface to MySQL to manage the presentation. Use Zope for the web interface and the RDBMS for the queries. On Sun, 23 Nov 2003, Andre Meyer wrote:
The application I am developing is for a musician who currently has an Excel sheet with more than 20.000 pieces of old music (1600-1900) where each PieceOfMusic object refers to other objects like Composer, OrchestralSetting, Instrument, Cast, Recording, Writer, etc. Those need to become different kinds of Python objects that the PieceOfMusic object can refer to in order to guarantee consistency. I have designed an object model with all the classes, their attributes and relations in a UML diagram, but now I need to convert the Excel sheet into Python objects in Zope and make them accessible through ZPT pages and Python methods based on this UML model.
I assume that I need a class MusicLibrary that is a direct subclass of ObjectManager and a variety of classes that represent my application classes and are all subclasses of ObjectManagerItem. Do you agree with this? In the help about ObjectManager and ObjectManagerItem I miss a method like manage_addObject. There are only manage_self, manage_addProduct and manage_addFolder. But which method does the trick of adding Python objects to my product's folder?
Andre Meyer wrote at 2003-11-23 10:15 +0100:
In continuing my search for help it seems like ObjectManager is what I am looking for in order to have a large collection of different Python objects in the same folder allowing for many links among them. I will need to be able to select which kinds of items to show and sort them and use them in forms. That should be standard behaviour, though, I would expect.
Have a look at the "ObjectManager" API (documented in Zope's online help system). The "nxm relations" (or something like that) may also be of help (not for the "ObjectManger" part but for the various relations). -- Dieter
For 20,000 objects you probably want BTreeFolder - it's a special version of the ObjectManager that handles large numbers of object more efficiently. I would create your new object by subclassing SimpleItem and CatalogAware (you will need to use the Catalog to index it) and then put these objects into the BTreeFolder. Check out Klassifieds on the Zope.org site, the application is fairly rubbish but it's very simple so hopefully easy to follow and follows the idea described here. A
On Sun, 2003-11-23 at 01:15, Andre Meyer wrote:
In continuing my search for help it seems like ObjectManager is what I am looking for in order to have a large collection of different Python objects in the same folder allowing for many links among them.
Possibly... though you may do better with Folder.
I will need to be able to select which kinds of items to show and sort them and use them in forms. That should be standard behaviour, though, I would expect.
It is, ex: stuff = objectValues(obj_type)
Do you happen to have experience with ObjectManager and/or have a good example that I can study?
Yep. Works like a charm. Check out Folder and other folderish objects in Zope (Plone, acl_users, etc).
The application I am developing is for a musician who currently has an Excel sheet with more than 20.000 pieces of old music (1600-1900) where each PieceOfMusic object refers to other objects like Composer, OrchestralSetting, Instrument, Cast, Recording, Writer, etc.
OK.
Those need to become different kinds of Python objects that the PieceOfMusic object can refer to in order to guarantee consistency.
I'm not so sure about that, but I follow your thinking.
I assume that I need a class MusicLibrary that is a direct subclass of ObjectManager
Yes... or better yet, of Folder. :-)
and a variety of classes that represent my application classes
Sure.
and are all subclasses of ObjectManagerItem.
Probably not. For node/leaf objects, you'll do better with SimpleItem.
Do you agree with this? In the help about ObjectManager and ObjectManagerItem I miss a method like manage_addObject.
Factory methods are defined by the objects they instantiate. Getting back to the object hierarchy... for each piece of music, you have (say) a Composer. If you define your piece of music as a folderish object that contains a composer object, you'll probably regret it. Why? Because it's equally as valid to use Composers as a container for specific compositions. How many times do you want to end up representing each composer, artist, etc? Once, I'd bet. Basically you have a many-to-many relationship between pieces and composers. In Zope, I find these are best modeled as objects in *peer* containers. Thus, you might have: MusicLibrary/ Composers/ Verdi Pieces/ Aida Otello The Verdi object would then have a "pieces" property of [Aida, Otello] and each of Verdi's operas has a "composers" property of [Verdi]. HTH, Dylan
At 02:14 PM 11/23/2003 -0800, Dylan Reinhardt wrote:
<much snipped> Basically you have a many-to-many relationship between pieces and composers. In Zope, I find these are best modeled as objects in *peer* containers. Thus, you might have:
MusicLibrary/ Composers/ Verdi Pieces/ Aida Otello
The Verdi object would then have a "pieces" property of [Aida, Otello] and each of Verdi's operas has a "composers" property of [Verdi].
While this is a straightforward M:N approach, it is not recommended for most applications because each piece of information is recorded twice, not once. Fact: Verdi wrote Aida. This fact should be recorded only once, not twice in two different places. This is standard practice when developing SQL database applications in order to (1) reduce storage needs and (2) to avoid the type of database error that can creep in any M:N storage problem where one side is updated or corrected (Verdi did not compoe "Hair") but the other part of the database is accidentally left uncorrected ("Hair" was composed by Verdi). When the data fact is stored only once, you avoid this type of problem... it is also easier to code updates to only one table than to two or more tables. That said, I have less experience developing M:N database structures in an object-oriented database like ZODB. There could be other reasons why you might indeed want such a structure... like speed. But that's what indexes are for, so I doubt that would be the reason. My leaning is toward Dennis's suggestion... use SQL. It seems a straightforward application there. Best wishes... =Paul
On Sun, 2003-11-23 at 17:39, Paul Howell wrote:
At 02:14 PM 11/23/2003 -0800, Dylan Reinhardt wrote:
The Verdi object would then have a "pieces" property of [Aida, Otello] and each of Verdi's operas has a "composers" property of [Verdi].
While this is a straightforward M:N approach, it is not recommended for most applications because each piece of information is recorded twice, not once.
An excellent point. If that's important to you, you could record it in only one place and the object hierarchy is still a good way to model this data. But a little redundancy *can* be a good thing in this domain, so long as it's managed well. In web apps, you often drill into data in arbitrary ways. Why should it be cheap to know who wrote Aida but expensive to know what else Verdi wrote? What's critical here is having *one* way of creating & changing the data. So, for example, you can create and edit composer objects independently, but can only assign composers to pieces in the piece management screens. That said, it may be best to think of each piece object as a large mapping for composer, artist, etc. Objects. If that streamlines development substantially, the extra cycles are well spent.
Fact: Verdi wrote Aida. This fact should be recorded only once, not twice in two different places.
More generally, you should have exactly one *process* for creating and maintaining each bit of data. There are any number of reasons one might choose to allow some redundancy in how data is accessed.
This is standard practice when developing SQL database applications
Understood. Zope at its best works almost nothing like an RDBMS.
My leaning is toward Dennis's suggestion... use SQL. It seems a straightforward application there. Best wishes...
You could certainly do that. If an RDBMS is the tool you're most familiar and comfortable with, that's probably the best solution for you. But this is *exactly* the area, IMO, where ZODB shows more power, intuition, and flexibility than SQL. It's not hard to understand why LAMP became the dominant method of creating web apps: it leveraged existing skills and assets. But Zope reflects much of what's been learned about web apps *after* LAMP caught on. You *can* use Zope as a high-octane PHP, but Zope is also capable of managing data in new and more interesting ways. If you're more comfortable with RDBMS/LAMP development, use that. But know that there is also value to breaking that mold. FWIW, Dylan
At 07:10 AM 11/24/2003 -0800, Dylan Reinhardt wrote:
On Sun, 2003-11-23 at 17:39, Paul Howell wrote:
At 02:14 PM 11/23/2003 -0800, Dylan Reinhardt wrote:
It's not hard to understand why LAMP became the dominant method of creating web apps: it leveraged existing skills and assets. But Zope reflects much of what's been learned about web apps *after* LAMP caught on. You *can* use Zope as a high-octane PHP, but Zope is also capable of managing data in new and more interesting ways.
If you're more comfortable with RDBMS/LAMP development, use that. But know that there is also value to breaking that mold.
FWIW,
Dylan
Thanks Dylan, I couldn't agree more. In particular, I like using an object-based data approach where there is a lot of disparate information to gather together about a certain topic... for example, information related to one customer (past orders, preferences, personal info other than sensitive types). Then a simple template without complex SQL queries can gather all the information at once from the Customer object for display when that customer is with you on the telephone (or chat/IRC/whatever). =Paul
Thanks a lot for all the comments. This helped me a lot and I made quite some progress. It is not impossible to use a RDBMS for this purpose and I have quite some experience with that kind of approach. However, one of the reasons to use Zope is that it is a much more advanced and truly OO environment. Therefore, why not use the power and flexibility built in Zope and add more administrative complexity using another tool like MySQL? The approach described by Dylan seems appropriate, but it is clearly a problem to have double references. That leads inevitably to inconsistency. Therefore, I will try the approach using Folders and SimpleItems, where different Folders are created for different kinds of SimpleItems (classes). All references will be one-way only. For now, I have added SimpleItems of different classes in the same Folder and used the class as selection criterion for building lists. This works very well for small amounts of items. Will it work well with many items? Will I have to use BTreeFolder (still need to have a look at its API)? Another issue at the moment is the referencing: I made a zpt page with a html selection form field where the values are the ids and the names are the titles of SimpleItems (composers in my case). This way, I can select any of the previously added composers and store its id in the PieceOfMusic object. The tricky part is how to retrieve the composer within a zpt page: using here/piece/composer/title in a tal:repeat does not work! How should this be handled? thanks and best wishes André ----- Original Message ----- From: "Paul Howell" <paul@smoothstone.com> To: "Zope Mailing List" <zope@zope.org> Sent: Tuesday, November 25, 2003 12:48 AM Subject: Re: [Zope] ObjectManager example needed
At 07:10 AM 11/24/2003 -0800, Dylan Reinhardt wrote:
On Sun, 2003-11-23 at 17:39, Paul Howell wrote:
At 02:14 PM 11/23/2003 -0800, Dylan Reinhardt wrote:
It's not hard to understand why LAMP became the dominant method of creating web apps: it leveraged existing skills and assets. But Zope reflects much of what's been learned about web apps *after* LAMP caught on. You *can* use Zope as a high-octane PHP, but Zope is also capable of managing data in new and more interesting ways.
If you're more comfortable with RDBMS/LAMP development, use that. But know that there is also value to breaking that mold.
FWIW,
Dylan
Thanks Dylan, I couldn't agree more. In particular, I like using an object-based data approach where there is a lot of disparate information to gather together about a certain topic... for example, information related to one customer (past orders, preferences, personal info other than sensitive types). Then a simple template without complex SQL queries can gather all the information at once from the Customer object for display when that customer is with you on the telephone (or chat/IRC/whatever). =Paul
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
On Tue, 25 Nov 2003, Andre Meyer wrote:
It is not impossible to use a RDBMS for this purpose and I have quite some experience with that kind of approach. However, one of the reasons to use Zope is that it is a much more advanced and truly OO environment. Therefore, why not use the power and flexibility built in Zope and add more administrative complexity using another tool like MySQL?
Andre, The mapping of your music spreadsheet database to RDBMS tables is straightforward if not trivial. Once done, you have the full relational query mechanism your application seems to need. Zope's integration with RDBMS systems like MySQL is exceptionally straightforward, clean, powerful, and flexible. You can then use Zope to handle the user interaction and presentation. If you choose to do the whole thing in Zope, you'll end up having to implement in Zope a subset of the relational query mechanism you'd have in a RDBMS. It'll probably be educational for you , the resulting product will be complex and difficult to maintain, it will take a lot of time and effort, and you'll have to worry about scalability and performance. If you choose the Zope front-end, RDBMS backend approach, you use minimize the amount of new mechanism that needs to be created and build on existing mechanism known to (mostly) work. You use tools in the domain where they work best--a RDBMS for selecting data to present based upon a query and Zope for data capture and presentation on the web.
On Tue, Nov 25, 2003 at 07:23:53AM -0800, Dennis Allison wrote:
If you choose to do the whole thing in Zope, you'll end up having to implement in Zope a subset of the relational query mechanism you'd have in a RDBMS. It'll probably be educational for you , the resulting product will be complex and difficult to maintain, it will take a lot of time and effort, and you'll have to worry about scalability and performance.
For *very* minimal but useful implementation of relations in zope, the mxmRelations product can be useful. You can't do complex queries but it can ease the pain of keeping track of related objects. However, it requires that your data objects and/or their containers inherit from a mixin class provided by this product.
If you choose the Zope front-end, RDBMS backend approach, you use minimize the amount of new mechanism that needs to be created and build on existing mechanism known to (mostly) work. You use tools in the domain where they work best--a RDBMS for selecting data to present based upon a query and Zope for data capture and presentation on the web.
I haven't actually tried it yet, but if it were me and I had the time to do some experimentation, I would look into using APE. As I understand it, this might be able to further improve the zope <-> rdbms integration: you could store the data heirarchically in the ZODB and use all of zope's nice features, and then when you want to do queries, use a zsql method to directly query the underlying relational storage. -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's SAMURAI BOXER! (random hero from isometric.spaceninja.com)
On Tue, 2003-11-25 at 07:23, Dennis Allison wrote:
The mapping of your music spreadsheet database to RDBMS tables is straightforward if not trivial.
Precious little is "trivial" in any significant RDBMS. Any time you're working on a system that executes dynamically-generated code, several issues loom large... maintenance surely among them.
Zope's integration with RDBMS systems like MySQL is exceptionally straightforward, clean, powerful, and flexible. You can then use Zope to handle the user interaction and presentation.
PHP is every bit as powerful and flexible but requires a far-smaller footprint. If you want to develop RDBMS-bound web apps, It's not clear what makes Zope the best tool for the job.
If you choose to do the whole thing in Zope, you'll end up having to implement in Zope a subset of the relational query mechanism you'd have in a RDBMS.
With Zope providing acquisition and arbitrary traversal, there's hardly anything left for you to do. Just make one object's id a property of another object. If you can find one from the other, you have all the relationship you'll ever need. Using my data model, let's say Aida has a "piece_composer" property of 'Verdi' and there is a Verdi object in the Composers folder, a peer to the Pieces folder. To render useful composer information in a Piece object's interface, all you need to do is something like: ----- <B>Composer</B>: <dtml-call "REQUEST.set('compsr_obj', _[piece_composer])"> <a href=Composers/<dtml-var "compsr_obj.getId()">
<dtml-var "compsr_obj.title"></a>
----- And you're done. Each piece displays its composer and links to the composer detail screen. Data management is practically transparent.
It'll probably be educational for you , the resulting product will be complex and difficult to maintain, it will take a lot of time and effort, and you'll have to worry about scalability and performance.
IMO, these are the leading reasons to choose ZODB development *instead* of RDBMS. I sell support contracts, so I'm only able to stay in business by achieving short development time & low TCO. Zope eats through a *lot* of technical resources, but it's pretty good about consuming human resources... the only kind I really care about. Having developed both ways, I'm not the least bit tempted to go back. YMMV, of course. I know mine does. :-) Dylan
On Tue, Nov 25, 2003 at 09:17:03AM -0800, Dylan Reinhardt wrote:
Using my data model, let's say Aida has a "piece_composer" property of 'Verdi' and there is a Verdi object in the Composers folder, a peer to the Pieces folder. To render useful composer information in a Piece object's interface, all you need to do is something like:
-----
<B>Composer</B>: <dtml-call "REQUEST.set('compsr_obj', _[piece_composer])"> <a href=Composers/<dtml-var "compsr_obj.getId()">
<dtml-var "compsr_obj.title"></a>
And you're done. Each piece displays its composer and links to the composer detail screen. Data management is practically transparent.
The more interesting problem is to find all pieces composed by Verdi. Fortunately, I can give the solution in one word: ZCatalog. you'd want your objects to inherit CatalogAware, of course. -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's NOT SO UGLY MASTICATOR! (random hero from isometric.spaceninja.com)
On Tue, 2003-11-25 at 10:19, Paul Winkler wrote:
On Tue, Nov 25, 2003 at 09:17:03AM -0800, Dylan Reinhardt wrote:
Using my data model, let's say Aida has a "piece_composer" property of 'Verdi' and there is a Verdi object in the Composers folder, a peer to the Pieces folder. To render useful composer information in a Piece object's interface, all you need to do is something like:
-----
<B>Composer</B>: <dtml-call "REQUEST.set('compsr_obj', _[piece_composer])"> <a href=Composers/<dtml-var "compsr_obj.getId()">
<dtml-var "compsr_obj.title"></a>
And you're done. Each piece displays its composer and links to the composer detail screen. Data management is practically transparent.
The more interesting problem is to find all pieces composed by Verdi.
Which is why my original suggestion included caching reverse references. Splitting a pair of "foreign keys" across two objects is not much more difficult than (or conceptually much different from) storing a pair of foreign keys in a third table/object. The important thing is that both elements of the pair be managed by a *single* set of processes. If the process that updates one side is the only process that updates the other side, you won't have to worry much about orphaned references. A small amount of validation can virtually eliminate the problem. Having a single process strongly implies using an *interface* for managing references... you don't want to simply expose your references in a text field. I recognize that this practice is unconventional, so I won't go very far out on a limb defending it. But it has worked very well for me and this list is all about sharing stuff that works well.
Fortunately, I can give the solution in one word: ZCatalog.
Sure, that works too... primarily by doing something very similar with a far greater amount of overhead. :-) Dylan
Andre Meyer wrote:
...
The approach described by Dylan seems appropriate, but it is clearly a problem to have double references. That leads inevitably to inconsistency. Therefore, I will try the approach using Folders and SimpleItems, where different Folders are created for different kinds of SimpleItems (classes). All references will be one-way only.
For now, I have added SimpleItems of different classes in the same Folder and used the class as selection criterion for building lists. This works very well for small amounts of items. Will it work well with many items? Will I have to use BTreeFolder (still need to have a look at its API)?
BTreeFolder should be a direct drop-in replacement for Folder. The get-by-meta_type function in Folder will probably degrade as your contents increase, but I don't know on what order, or if BTreeFolder would do better. If you're going to have a stupendous amount of objects, then ZCatalog is the way to go.
Another issue at the moment is the referencing: I made a zpt page with a html selection form field where the values are the ids and the names are the titles of SimpleItems (composers in my case). This way, I can select any of the previously added composers and store its id in the PieceOfMusic object. The tricky part is how to retrieve the composer within a zpt page: using here/piece/composer/title in a tal:repeat does not work! How should this be handled?
That's to broad of a problem to answer without a lot of supposition, and my mind-reading skills seem to be failing on this one. You'll have to be a little more specific on your structure and goal (and code). --jcc -- "Code generators follow the 80/20 rule. They solve most of the problems, but not all of the problems. There are always features and edge cases that will need hand-coding. Even if code generation could build 100 percent of the application, there will still be an endless supply of boring meetings about feature design." (http://www.devx.com/java/editorial/15511)
Andre Meyer wrote:
I assume that I need a class MusicLibrary that is a direct subclass of ObjectManager and a variety of classes that represent my application classes and are all subclasses of ObjectManagerItem.
ObjectManagerItem is not a real class, and seems to have been invented for the documentation. You really want SimpleItem, I suspect.
Do you agree with this? In the help about ObjectManager and ObjectManagerItem I miss a method like manage_addObject. There are only manage_self, manage_addProduct and manage_addFolder. But which method does the trick of adding Python objects to my product's folder?
Zope asks each product to provide factories to add themselves. You get access to the registered factory methods (like manage_addFolder) with manage_addProduct. http://www.zopelabs.com/cookbook/1012279676 mxmEasyProduct makes product bootstrap easier. mxmRelations may help your "links." But you might consider using CMF or Plone so that you can use Archetypes, which makes development of content types (which is really what you have) really dead easy. And it also provides some very nice reference machinery. For finding objects in Zope where the amount or location is non-trivial, you will usually use ZCatalog. --jcc -- "My point and period will be throughly wrought, Or well or ill, as this day's battle's fought."
Hi Andre, Andre Meyer wrote:
The application I am developing is for a musician who currently has an Excel sheet with more than 20.000 pieces of old music (1600-1900) where each PieceOfMusic object refers to other objects like Composer, OrchestralSetting, Instrument, Cast, Recording, Writer, etc. Those need to become different kinds of Python objects that the PieceOfMusic object can refer to in order to guarantee consistency. I have designed an object model with all the classes, their attributes and relations in a UML diagram, but now I need to convert the Excel sheet into Python objects in Zope and make them accessible through ZPT pages and Python methods based on this UML model.
This sounds like it might be an equally good (or possibly better...) fit for an RDBMS solution. Is there a reason why this is not an option for you? -- Regards, PhilK Email: phil@xfr.co.uk / Voicemail & Facsimile: 07092 070518 "the symbols of the divine show up in our world initially at the trash stratum." Philip K Dick
participants (11)
-
alice -
Andre Meyer -
Andrew Veitch -
Dennis Allison -
Dieter Maurer -
Dylan Reinhardt -
J Cameron Cooper -
J. Cameron Cooper
-
Paul Howell -
Paul Winkler -
Philip Kilner