Using Zope for Groupware/Messaging applications
Hi! We are currently developing a groupware system with Zope. As usual, the options for the storage back-end are using a SQL DB (postgres, ...) or ZODB. I know that this question has been asked a thousand times. But I'll repeat it for our specific situation: SQL or ZODB+ZCatalog, which one is better? We have thousands, maybe millions of objects (mainly mail messages, notes, appointments). There should be very flexible views on the data, e.g. a message or document can either be viewed on the owner's personal workspace or on the project portal the document was posted to. The problems with SQL: - much more complex setup - problems with using ZEO - BLOBs (large binary objects like image files) don't work well on most databases The problems with ZODB/ZCatalog: - ZCatalog is not tested for extremly large numbers of objects. We fear that the index tables will become much too big - Some advanced queries are very hard to implement using ZCatalog Any comments? Joachim.
R. David Murray wrote:
On Sat, 17 Feb 2001, Joachim Werner wrote:
SQL or ZODB+ZCatalog, which one is better?
Any comments?
Just the obvious one: ZPatterns. <grin>.
OK, this was the standard answer, right. But what I don't really get is: How can ZPatterns help me here? If My underlying storage can't do what I want it to do (let's say very expensive queries -- which is what ZCatalog can't really do well), how can an abstraction layer on top of it solve my problems? And on the other hand, if I only have one option that does what I want (let's say using SQL) and I know that in advance, why should I use ZPatterns then? And, instead of switching back-ends in ZPatterns, I could as well just rewrite my non-ZPattern objects to work with the new back-end, which doesn't seem to be more work. I mean, if I switch from using ZClasses/ZCatalog to ZSQL in ZPatterns, I still have to rewrite all my methods that handle storage and data retrieval. ZPatterns don't do that for me. I have read and seen a lot of ZPatterns stuff, but I still haven't seen something where I really NEED ZPatterns to get a better solution for a problem. Cheers, Joachim
Joachim Werner wrote:
And, instead of switching back-ends in ZPatterns, I could as well just rewrite my non-ZPattern objects to work with the new back-end, which doesn't seem to be more work.
Well, do your first project both ways. It would make an interesting comparative study. I think you're getting distracted by the flexible storage options of ZPatterns, and missing the main benefit to your application. In your original message, you said:
There should be very flexible views on the data, e.g. a message or document can either be viewed on the owner's personal workspace or on the project portal the document was posted to.
"Flexible views on the data" is where ZPatterns really makes things easier. Of course there are other ways to achieve the same ends. However, I believe that using ZPatterns makes it easier to evolve your system in the face of changing requirements. -- Steve Alexander Software Engineer Cat-Box limited http://www.cat-box.net
Let me be a bit less cryptic. Design your system, and use ZPatterns, and you can defer questions about which storage suits the parts of the app until later, and can change your mind at need. For the kind of project you are embarking upon, the time needed to learn ZPatterns is well worth it. --RDM
"R. David Murray" wrote:
Let me be a bit less cryptic. Design your system, and use ZPatterns, and you can defer questions about which storage suits the parts of the app until later, and can change your mind at need. For the kind of project you are embarking upon, the time needed to learn ZPatterns is well worth it.
--RDM
I concur. It would be best to abstract the storage. Using this technique you could scale the underlying storage theoretically without changing the top level code. That would also facilitate more experimentation on which scheme or combination of schemes would work best. -- | Casey Duncan | Kaivo, Inc. | cduncan@kaivo.com `------------------>
"R. David Murray" wrote:
Let me be a bit less cryptic. Design your system, and use ZPatterns, and you can defer questions about which storage suits the parts of the app until later, and can change your mind at need. For the kind of project you are embarking upon, the time needed to learn ZPatterns is well worth it.
I've yet to see any documentation which sufficiently explains what ZPatterns is and how to use it for me to consider it in any of my projects. I've read the wiki pages and I just don't get it. Where should I be starting for enlightenment? Doug
On Sat, 17 Feb 2001, Doug Hellmann wrote:
I've yet to see any documentation which sufficiently explains what ZPatterns is and how to use it for me to consider it in any of my projects. I've read the wiki pages and I just don't get it.
Where should I be starting for enlightenment?
Well, if you've read the Wiki, there really isn't anything else to read other than the Coad book for general object design background. I "figured out" zpatterns by starting to build a store using EMarket 0.2.0, which is ZPatterns based. It didn't take very long before I started to really appreciate the power of the ZPatterns approach. Fundamentally think of it as doing the object design of your system without worrying about where the data is going to get stored. If you want a nuts and bolts capsule description, think of it as providing you the ability to easily make just about anything about your objects computed. That is, when any part of your application goes to access one of your objects, there is code that can get triggered to do all sorts of Cool Stuff in order to build the object from arbitrary data sources. This means you can do things like have (my example) a Paradox database of books, authors, and a books-to-authors map, and have Author and Book objects in the system such that a Book has a list of its Author objects and the Author has a list of its Book objects, all in about 20 lines of ZSQL and skinscript. Plus I have a 'long description' field that comes from another database table, and thumbnail and full sized images that are stored in Image objects in the ZODB. All this stuff is seemlessly assembled into Author and Book objects by, like I said, about 20 lines of the skinscript and zsql. The rest of the app has no idea how those objects come to be or where the data comes from (or goes to). You really have to try it to understand, though, I think. It's not as hard as the discusions on the mailing list may make it seem. The learning curve is somewhat steep, but also rather short, at least for the wow-this-is-powerful stuff (I'm still learning the subtleties). --RDM
"R. David Murray" wrote:
This means you can do things like have (my example) a Paradox database of books, authors, and a books-to-authors map, and have Author and Book objects in the system such that a Book has a list of its Author objects and the Author has a list of its Book objects, all in about 20 lines of ZSQL and skinscript. Plus I have a 'long description' field that comes from another database table, and thumbnail and full sized images that are stored in Image objects in the ZODB. All this stuff is seemlessly assembled into Author and Book objects by, like I said, about 20 lines of the skinscript and zsql.
How are you entering the data into the system? Specifically, how are you *normalizing* the Author and book information? I'm working on a ZPatterns based book cataloging application, and right now I'm storing 'Book' objects with the author names as a 'lines' property in the ZClass. I'd like to change this so 'Author' objects are created and stored in their own Specialist when a book is added, but I'm not sure how. Thanks, Michael Bernstein.
On Sun, 18 Feb 2001, Michael R. Bernstein wrote:
How are you entering the data into the system? Specifically, how are you *normalizing* the Author and book information?
I don't know what you mean by normalizing, I'm afraid. The data is stored and upated in the external Paradox database, which I then import into PostgreSQL using a Paradox-to-SQL converter and some bash scripts.
I'm working on a ZPatterns based book cataloging application, and right now I'm storing 'Book' objects with the author names as a 'lines' property in the ZClass.
I'd like to change this so 'Author' objects are created and stored in their own Specialist when a book is added, but I'm not sure how.
I'm not sure I understand what the issue is. Can't you just, when data is entered to create a Book, query the Authors specialist to see if the Author exists and create it if it doesn't? --RDM
Hi, I have the need to make a group extension to the permission system in Zope. I need to make so kind of folder or acl_extender that can know about groups of users and map them to local roles. As I understand the security system user get their role mapping through a UserManager (acl_users). But what I want is to have a central UserManager (possibly LoginManager or using the PTK member system), but on a branch level extend the it with user-group-role mapping. Any ideas on how to solve this problem? Maybe someone already have thought about this? I anticipate using ZPatterns to do the mapping separating the storage from the logic. Regards, Johan Carlsson
participants (7)
-
Casey Duncan -
Doug Hellmann -
Joachim Werner -
Johan Carlsson -
Michael R. Bernstein -
R. David Murray -
Steve Alexander