ZODB or MySQL for form data?
I'm coming to Zope from PHP where my database is MySQL. However I am wanting to move to Zope and am trying to learn the correct way of working in this platform. I know that I can connect to MySQL from Zope but from what I read it appears that the use of this feature is recommended only if one is wanting to use an existing database. Since I am starting a new Zope application, and will be creating new data files, is it better to use MySQL or ZODB to handle form data? I would really appreciate hearing from experienced Zope users on this matter. Paul
On Mon, 08.09.2003 at 11:32 -0400, paulporter@buffalotalks.com wrote:
is it better to use MySQL or ZODB to handle form data?
I would really appreciate hearing from experienced Zope users on this matter.
I would recommend ZODB. But then again, I'm not experienced. :) -- paavo. Proletarier aller Länder, macht Schluß!
Paul, The answer to this question depends a lot on the kind of data that you are working with, what experience you have and the time you have to (re)implement. Our experience has been to store 90-95% of our data in a backend db (postgres for us) and the rest is stored in zope. This has worked exceptionally well for us. Zope then contains most of our business logic and all presentation logic. We have stayed away from storing everything in zope because some things that are easy in sql are still cumbersome using zope objects. Complex selects with joins and mass updates, deletions come to mind. This is especially true if you have large numbers of objects and changing complex select criteria. Also, it has worked well for us to have this division of data and presentation when replicating data. But, you do pay a performance hit for each db hit. Cacheing can be used in zope to help reduce this expense in certain cases. On the other hand, replicating those pieces of data stored in zope between our dev and live servers has been a manual and somewhat painfull process for us. Mainly because we have to be sure that when we push new presentation or logic, we don't overwrite data on the live server. And someone has to remember to push the data pieces back to our dev server periodically. So, my opinion is this. One needs to weigh one's needs and the problem domain with the tools you have available to you. If you are experienced in SQL and already have an application using sql extensively, you will have a working application much faster if you use zope as a presentation engine and keep your db in MySql. As your familiarity increases, you may find that moving some or all of your app into zope might give you benefits that using a sql backend does not. OO programming of course has some benefits that are difficult to replicate in sql (try doing trees in SQL for example). The opposite of course is true as well. Oh, and by the way, welcome. I migrated from PHP to zope a number of years back as well. Good luck. My $.02 -Chris On Monday 08 September 2003 11:32 am, paulporter@buffalotalks.com wrote:
I'm coming to Zope from PHP where my database is MySQL. However I am wanting to move to Zope and am trying to learn the correct way of working in this platform. I know that I can connect to MySQL from Zope but from what I read it appears that the use of this feature is recommended only if one is wanting to use an existing database. Since I am starting a new Zope application, and will be creating new data files, is it better to use MySQL or ZODB to handle form data?
I would really appreciate hearing from experienced Zope users on this matter.
Paul
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
-- Chris Kratz Systems Analyst/Programmer VistaShare LLC www.vistashare.com
At 11:32 AM 9/8/2003, paulporter@buffalotalks.com wrote:
[...] Since I am starting a new Zope application, and will be creating new data files, is it better to use MySQL or ZODB to handle form data?
I think the ZODB is best suited to holding larger parts of your Web site, like entire Web pages, parts of Web pages (headers, footers), scripts, and at the smaller end images, etc. I think tiny pieces of information like single records of form data are better held in a relational DB like MySQL. Consider this: do you really want to have a single, independent Zope object for every record of submitted form data? You will quickly have Zope folders with hundreds of items (listed alphabetically). Such data records would be much easier to sort, access, and manipulate in a relational DB like MySQL or PostgreSQL. Hope this helps and good luck, Jon Whitener Detroit Michigan USA
On Mon, 2003-09-08 at 10:32, paulporter@buffalotalks.com wrote:
I'm coming to Zope from PHP where my database is MySQL. However I am wanting to move to Zope and am trying to learn the correct way of working in this platform. I know that I can connect to MySQL from Zope but from what I read it appears that the use of this feature is recommended only if one is wanting to use an existing database. Since I am starting a new Zope application, and will be creating new data files, is it better to use MySQL or ZODB to handle form data?
I would really appreciate hearing from experienced Zope users on this matter. [snip]
Depends. The ZODB is an object database with a tree like hierarchy. So if you data fits well into that model feel free. If you do use ZODB you may want to develop some objects in which to store your data. If you data is highly relational something like Postgres or Mysql is probably better. In the end it depends on what you are familiar with and what the data is. You can even mix the two. :-) -- Edward Muller Interlix (http://www.interlix.com) Phone: 417-862-0573 - Cell: 417-844-2435 - Fax: 417-862-0572 Web Hosting - PC Service & Support - Custom Programming - Network Service & Support Specializing in Open Source Solutions
I use MySQL as a backend database because my application needs SQL and the infrastructure that comes with it. It's stable and clean. The site also uses the ZODB to store some kinds of information which is independent of the presentation. IMHO, the best approach is to use tools which match the needs of the application. On Mon, 8 Sep 2003 paulporter@buffalotalks.com wrote:
I'm coming to Zope from PHP where my database is MySQL. However I am wanting to move to Zope and am trying to learn the correct way of working in this platform. I know that I can connect to MySQL from Zope but from what I read it appears that the use of this feature is recommended only if one is wanting to use an existing database. Since I am starting a new Zope application, and will be creating new data files, is it better to use MySQL or ZODB to handle form data?
I would really appreciate hearing from experienced Zope users on this matter.
Paul
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
I'm coming to Zope from PHP where my database is MySQL. However I am wanting to move to Zope and am trying to learn the correct way of working in this platform. I know that I can connect to MySQL from Zope but from what I read it appears that the use of this feature is recommended only if one is wanting to use an existing database. Since I am starting a new Zope application, and will be creating new data files, is it better to use MySQL or ZODB to handle form data?
I would really appreciate hearing from experienced Zope users on this matter.
I'm a bit on the other side of the fence from most of the rest of the folks who have replied. My take on the matter is that unless you have a very compelling reason or need to use an external RDBMS you should keep everything in Zope. For one thing, the fewer different modes of operation you have the better. (I put a lot of value in not having to deal with a separate relational database, from the point of view of complexity, reliability, performance, and administration.) And although the integration of relational data into Zope is very good, it's not quite like having your data available as real live objects in what is, after all, a program. I tend to get a more mileage out of in-Zope data than imported data. But then, I started out with Zope, rather than DB-backed scripting like PHP, and am a programmer first and web developer second. And other people may not be quite as allergic to attaching a database to Zope as I am. In the end, of course, it depends on what you're doing. A real discussion, rather than general hot-air, would depend on the problem domain. --jcc -- "My point and period will be throughly wrought, Or well or ill, as this day's battle's fought."
On Monday, September 8, 2003, at 05:31 PM, J Cameron Cooper wrote:
I'm a bit on the other side of the fence from most of the rest of the folks who have replied. My take on the matter is that unless you have a very compelling reason or need to use an external RDBMS you should keep everything in Zope.
I'm curious as to how to best find information when storing data in Zope. I come from a RDBMS background, so writing SQL statements to find the data that matches a user's request is a no-brainer for me. I would be interested in how one would store hundreds or thousands of records of data in Zope, and then find just the one or two that match the given criteria. Do you have to iterate through the entire branch of the ZODB, or is there another, simpler way that I'm missing? ___/ / __/ / ____/ Ed Leafe http://leafe.com/ http://opentech.leafe.com
Do you have to iterate through the entire branch of the ZODB, or is there another, simpler way that I'm missing?
ZCatalog indexes and allows efficient searching of content for you. Have a look in the Zope Book for more information. -- Andy McKay http://www.agmweb.ca
I'm a bit on the other side of the fence from most of the rest of the folks who have replied. My take on the matter is that unless you have a very compelling reason or need to use an external RDBMS you should keep everything in Zope.
I'm curious as to how to best find information when storing data in Zope. I come from a RDBMS background, so writing SQL statements to find the data that matches a user's request is a no-brainer for me. I would be interested in how one would store hundreds or thousands of records of data in Zope, and then find just the one or two that match the given criteria. Do you have to iterate through the entire branch of the ZODB, or is there another, simpler way that I'm missing?
As mentioned previously, if you're really doing that, then you go ZCatalog. However, sometimes (or often, depending on your problem domain) you can exploit the properties of the object database to make your searching implicit. For example, a directory (like ODP [dmoz.com] or Yahoo) implemented in relational data would endlessly be asked to find things with a certain category key or in a certain range. Or maybe you'd be doing some clever joining or whatnot. In Zope you could use containment to store relationships and position, and forthe contents of a certain category list the contents of that container rather than finding all rows with that container's key. Applications like this are pretty common in web apps. However, if your application really is a great flat pool of data, with a bunch of highly variable mostly arbitrary searches, then you may consider using an RDBMS. That is, after all, what they're really good at doing. Just be sure that the way you think your data domain is structured really is how it should be structured. --jcc -- "My point and period will be throughly wrought, Or well or ill, as this day's battle's fought."
On Monday 08 September 2003 05:36 pm, Ed Leafe wrote:
On Monday, September 8, 2003, at 05:31 PM, J Cameron Cooper wrote:
I'm a bit on the other side of the fence from most of the rest of the folks who have replied. My take on the matter is that unless you have a very compelling reason or need to use an external RDBMS you should keep everything in Zope.
I'm curious as to how to best find information when storing data in Zope. I come from a RDBMS background, so writing SQL statements to find the data that matches a user's request is a no-brainer for me. I would be interested in how one would store hundreds or thousands of records of data in Zope, and then find just the one or two that match the given criteria. Do you have to iterate through the entire branch of the ZODB, or is there another, simpler way that I'm missing?
Create a very simple python product to hold the data structure you need and make it catalog path aware. Store it in a BTree folder so you can have hundreds of thousands of items in a folder with no problem. Since the items are catalog aware they will remain correct in the catalog on any change to them so you can query the catalog for the data you want just set up what fields you want to index. The whole thing is an OODB so design it like you would an in memory app for an OO program. Make your data structure, index what you want from it for faster access, then use that index to filter for the data you want. This approach works well in general however in a lot of cases there are more specific methods that will outperform this one by a huge margin. For example if your company has a hierarchy of products and you want to find specific types it is often better to store it in that hierarchy. Example you have a games hierarchy and inside that you have action, rpg, strategy, sim, etc and then you may break it up further. You can then do a search in the zcatalog on a pathindex for games games/action etc to get all the items in that group. Also while zope presents the db as hierarchical it really is not. If you want an item to be in multiple places without copying it is trivial to do from a python product. You just store the object reference like your normally would in a python program just make sure to remove the acquisition wrapper before storage. This gives you one real copy of the object in the db but in many locations. If you often perform x type of query design a data structure that works well for that type of query of that query is expensive to make. Then use something like an observer pattern to keep that data structure always up to data as changes as made. What you end up with is something like caching a search result as far as speed goes but it is all completely dynamic since you update the data structures real time. OODBs are harder in some ways to design for then an RDB but with a good design you can often make them run far faster.
On Mon, 2003-09-08 at 22:45, kosh wrote:
On Monday 08 September 2003 05:36 pm, Ed Leafe wrote:
On Monday, September 8, 2003, at 05:31 PM, J Cameron Cooper wrote:
I'm a bit on the other side of the fence from most of the rest of the folks who have replied. My take on the matter is that unless you have a very compelling reason or need to use an external RDBMS you should keep everything in Zope.
I agree 100%, just on the merits of the ease of maintenance when your data is in Zope. Stuff like custom content types and Archetypes makes it almost automatic to generate data entry/updating screens. SQL-based data has to have interfaces built from scratch... Which brings me to a related question: Imagine having the standard theoretical employee table in a RDBMS. Other apps use it, so you can't put it into Zope. What is the easiest and most efficient method to provide data manipulation from within Zope? For example, could you map an SQL query to a content type? If so, would it be able to handle updates? And how would relational fields be mapped? -- Mitch
On Tue, Sep 09, 2003 at 10:15:16AM -0400, Mitch Pirtle wrote:
Which brings me to a related question:
Imagine having the standard theoretical employee table in a RDBMS. Other apps use it, so you can't put it into Zope. What is the easiest and most efficient method to provide data manipulation from within Zope?
APE might be interesting. - Paul Winkler http://www.slinkp.com Look! Up in the sky! It's BURNING PHYSICIAN SLIDE LOCK! (random hero from isometric.spaceninja.com)
Shane Hathaway's article "Evaluation of the Need for Better Zope-RDBMS Integration" may shed some light on this thread. http://hathaway.freezope.org/Writings/ZopeRDBMSResearch.html At 9/8/2003 11:32 AM, paulporter@buffalotalks.com wrote:
[edit] Since I am starting a new Zope application, and will be creating new data files, is it better to use MySQL or ZODB to handle form data?
Along with everyone's take: A simeple rule (maybe too simple) would be if it is highly relational data put in a SQL database and call it. If it is better nested (object oriented data) that could benefit from all of the security, through the web editing and other such fine Zope features put it in Zope. EX: NFL data, teams, players, seasons = SQL EX: Directory of addresses, locations, shops = ZODB There are also things that Zope can and will handle better with Zope products that you would have to build from scratch if you used an SQL database and I think that is why they say to use it if you already have it. The other consideration is size... while Zope can handle a TON of data (objects) if you are talking millions of items (rows) you might want to throw it in a SQL database. BZ
I'm coming to Zope from PHP where my database is MySQL. However I am wanting to move to Zope and am trying to learn the correct way of working in this platform. I know that I can connect to MySQL from Zope but from what I read it appears that the use of this feature is recommended only if one is wanting to use an existing database. Since I am starting a new Zope application, and will be creating new data files, is it better to use MySQL or ZODB to handle form data?
I would really appreciate hearing from experienced Zope users on this matter.
Paul
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
participants (13)
-
Andy McKay -
BZ -
Chris Kratz -
Dennis Allison -
Ed Leafe -
Edward Muller -
J Cameron Cooper -
Jon Whitener -
kosh -
Mitch Pirtle -
Paavo Parkkinen -
Paul Winkler -
paulporter＠buffalotalks.com