RE: [Zope] To Zope or Not To Zope?
Ryan wrote:
A client of ours has approached us about designing and building a major web portal project focused around a particular topic area with extensive member services and content management abilities. We now have to spec this project out and at some point in the near future I will have to decide on a WWW server/Application server/Database combination. Zope is in the running, along with WebObjects, NetObjects, ODI's ObjectStore + ObjectForms, etc.
That's great to hear that we're being considered with such an auspicious (if expensive :^) lineup!
My initial impulse is to go with Zope because of its low cost overhead and my expertise with Python as well as the access I have to this mailing list and so on. Unfortunately my desire to do implement python is offset by a few major concerns, and I would like to see if anybody out there could address these questions/concerns:
You've definately done a straightforward job of delineating the concerns. Thanks for the evenhanded treatment.
1) Zope seems to want to operate in a configuration of web server,
Hmm, I disagree a bit here. I still recommend people use Apache.
application server, and object database all in one package. I have managed to get Zope to run using ZServer + Apache PCGI, but the performance seems quite quite dismal. My concern is mainly
We certainly haven't seen dismal performance. Could you be more specific? I think you should expect a million hits per day on a $3k machine using Apache in front of Zope. If you're talking an order of magnitude more than that, then you might be right. But that would make your site one of the top ten Internet destinations :^) To be more specific, if you'd like a significant boost in Apache performance, write a mod_zope that eliminates the forking and pcgi-wrapper connection. Or contract with us to do it and put it back out in the community.
scalability: I want to be able to use an industry standard web server (i.e. apache) separate from the application server so that I could do things like scale using multiple web servers, etc. etc.
There are a whole number of reasons to want to keep Apache in the mix. We run Apache on zope.org due to this.
2) The same issue applies to the DB layer. After floating through all the documentation on the zope.org site I am unclear what the relationship would be between Zope and an industrial strength database like Oracle 8, etc. I am wary of using Zope's own object DB because of
It's certainly legitimate in these remaining weeks before Zope2 to fret about scale and the object database. We'll have a lot more confidence once Z ODB 3 comes out of testing. Until then, you're raising a legitimate point. As for compatibility, I realize that some people don't feel comfortable "locking" their database in a Data.bbb file. We're taking steps to make this decision easier (e.g. XML import/export, Zope2's ability to store its contents in one or more relational database tables, etc.). But those won't be ready for use for at least another month.
scale and compatibility issues, but most of the benefits of using Zope would be mitigated if I ended up having to write SQL queries
Correct.
to access my Oracle database. The question is: is it possible to use a DB like Oracle or Postgres, etc. transparently and in the same manner that I would be using Zope's native db? How about an object DB like ODI's ObjectStore?
Right, the idea for Z ODB 3 is that you can have multiple storage managers as part of one logical object system. The object system could be partitioned across three files, a Sybase table, and an Oracle table. It's likely that the part that stores data in the relational database might start out commercial, as well as the client-server version of the object database.
3) Zope is not "standard" enough that should I get run over by a truck development could proceed without me. Finding consultants and programmers other than myself to work on this project could be difficult.
I completely realize that going with Zope is still a risky decision. However, the question of risk due to being non-standard is mostly offset by the risk of going with a closed-source vendor. Do you _really_ trust Apple et al. to be responsive to all of your issues? IMO, customers should demand control. Again, thanks a bunch for taking the time to write up a well-reasoned statement of the issues! Paul Everitt Digital Creations paul@digicool.com 540.371.6909 ----------------------------------------- The Open Source Zope application server http://www.zope.org/ -----------------------------------------
Paul Everitt wrote: [snip]
application server, and object database all in one package. I have managed to get Zope to run using ZServer + Apache PCGI, but the performance seems quite quite dismal. My concern is mainly
We certainly haven't seen dismal performance. Could you be more specific?
Actually, the performance on NT/IIS is pretty dismal, too -- it's all in the fact you fork a new process for every friggin request. ISAPI on NT and IIS [and others] and a mod_zope for Apache would be extremely nice. [snip]
As for compatibility, I realize that some people don't feel comfortable "locking" their database in a Data.bbb file. We're taking steps to make this decision easier (e.g. XML import/export, Zope2's ability to store its contents in one or more relational database tables, etc.). But those won't be ready for use for at least another month.
It isn't just an issue of compatibility, but also of speed, stability, and scalability. Being able to have the Zope piggyback an MSSQL, DB2, or Oracle server is definitely a huge comfort, because a _lot_ of man hours -- arguably more than in the case of Zope -- have been put into those RDBMS tweaking them and securing them. Instead of spending lots of time finding new, esoteric ways of optimizing Zope's DB, you can effectively delegate that work to people who have spend their whole lives with their heads inside database algorithms. (Although I admit that B-trees are nice.)
scale and compatibility issues, but most of the benefits of using Zope would be mitigated if I ended up having to write SQL queries
Correct.
I don't agree (if the above sentence means that having write SQL queries will diminish the benefits of using Zope). SQL Methods greatly reduce the gap between the RDMS and Zope. It doesn't magically objectify the "relational" bits, but at least integrates them pretty well into the Zope object space.
It's likely that the part that stores data in the relational database might start out commercial, as well as the client-server version of the object database.
This statement piques me, for several reasons. I don't mind that Digital Creations publish products that are commercial and cost $, but I'd object to any Microsoft-ish pricing (eg., if you'd sold Confera for $1000 I'd definitely balk). So please be lenient, huh? But more than that I'd like to ask you what happens when you demote a commercial product to a free one. Isn't this unfair to all those people who might have bought it? Maybe you should consider stratifying your product line into "free" and "subscription". If I join the Zope Gold Club for $500/year, I get all your heavy-duty client/server stuff for free. Sort of a "fixed rate" thing.
3) Zope is not "standard" enough that should I get run over by a truck development could proceed without me. Finding consultants and programmers other than myself to work on this project could be difficult.
I completely realize that going with Zope is still a risky decision. However, the question of risk due to being non-standard is mostly offset by the risk of going with a closed-source vendor. Do you _really_ trust Apple et al. to be responsive to all of your issues? IMO, customers should demand control.
The risk from my own point of view is really that Zope's design suddenly one day might prove inefficient. For example, what if Zope doesn't scale beyond 1000 objects? That's my fear, because such a break in scalability may be deep in its design. I doubt that Zope doesn't handle 1000 objects, because obviously you've done some testing beyond that limit (you have, haven't you? Right? RIGHT? :), but since I've not looked through all the sources I can never be sure that at some point I'll say "I'd like to do..." and somebody will tell me "You can't, because internally Zopes this and that as...". That's how Windows tends to work. :-) Alexander Staubo mailto:alex@mop.no http://www.mop.no/~alex/
Alexander Staubo wrote:
This statement piques me, for several reasons. I don't mind that Digital Creations publish products that are commercial and cost $, but I'd object to any Microsoft-ish pricing (eg., if you'd sold Confera for $1000 I'd definitely balk). So please be lenient, huh? But more than that I'd like to ask you what happens when you demote a commercial product to a free one. Isn't this unfair to all those people who might have bought it?
Maybe you should consider stratifying your product line into "free" and "subscription". If I join the Zope Gold Club for $500/year, I get all your heavy-duty client/server stuff for free. Sort of a "fixed rate" thing.
Digicool is offering to give you stuff for free, and you want to pay for it? I think you're kinda missing the point of the OpenSource model - selling support, expertise, and "the feature you requested goes in FIRST since you paid", not software as such. The problem with Microsoft isn't their prices - it's that their software is closed and proprietary. -- Itamar - itamars@ibm.net -----------------------------o-------------------------------------o Sealingwax Greeting Cards | The only good morning is a dead one | http://www.sealingwax.com/ | --Richard Stallman |
On Wed, 26 May 1999, Alexander Staubo wrote:
Paul Everitt wrote: [snip] [snip] [snip] [snip]
It's likely that the part that stores data in the relational database might start out commercial, as well as the client-server version of the object database.
This statement piques me, for several reasons. I don't mind that Digital Creations publish products that are commercial and cost $, but I'd object to any Microsoft-ish pricing (eg., if you'd sold Confera for $1000 I'd definitely balk). So please be lenient, huh? But more than that I'd like to ask you what happens when you demote a commercial product to a free one. Isn't this unfair to all those people who might have bought it?
Digital Creations have made their standpoint on this clear several times, and personally I have to agree with them. If they make a product available for a price, then in paying that you are paying to get it NOW. With most products, including your example of Windows, you pay just to get your hands on it. If the documentation is not yet complete, or certain areas of procedure have yet to solidify, then to release it would generate a wave of requests for assistance, which DC cannot afford to supply. Therefore, those products whose current state is one which can be expected to generate a more than reasonable amount of 'need' in terms of support and care, then they restrict its distribution by making it available only to those who need it enough to pay for it, thereby limiting its spread (and the resulting calls for help) and providing sufficient remuneration that DC can afford to answer those queries which do appear. Witness ZTables, which has been available for some time, and in use in several places, as a commercial, i.e. remuneration-required, product. When it reaches a state of easy usability and sufficient documentation, it will become available to those who do not have a strong enough 'need' to pay for it upfront. They don't demote a commercial product to a free one. They promote it to an open one! Their opening of the code has benefits even for those who initially purchased it, in terms of a wider body of knowledgable developers who can answer questions regarding it, etc. Think of it in reverse. What is really happening, is that they are developing a free/open product, but can't afford to release it until they can handle the volume of traffic that it will generate. You are paying them to release it early to you. I know that personally, with respect to their stated goal of maintaining an arm's length, the picture of a stratified community, with some relegated to the 'free' club and others given special treatment in the 'gold' club seems likely to generate strain betwixt and between that is not necessarily present in the other view. Basically, we are all on a level playing field... the information and tools are available to all, if you wait long enough, and for those who can't wait, a bit of money will grease the wheels. Let's not divide the community. Paul, DC, let me just say once again, that in all the reading of articles and discussions and responses, never have I seen a negative image of Zope or DC go by. You guys are on target in maintaining the proper relationship with the community you have fostered and built around Zope. Keep up the excellent work.
Maybe you should consider stratifying your product line into "free" and "subscription". If I join the Zope Gold Club for $500/year, I get all your heavy-duty client/server stuff for free. Sort of a "fixed rate" thing.
3) Zope is not "standard" enough that should I get run over by a truck development could proceed without me. Finding consultants and programmers other than myself to work on this project could be difficult.
I completely realize that going with Zope is still a risky decision. However, the question of risk due to being non-standard is mostly offset by the risk of going with a closed-source vendor. Do you _really_ trust Apple et al. to be responsive to all of your issues? IMO, customers should demand control.
The risk from my own point of view is really that Zope's design suddenly one day might prove inefficient. For example, what if Zope doesn't scale beyond 1000 objects? That's my fear, because such a break in scalability may be deep in its design. I doubt that Zope doesn't handle 1000 objects, because obviously you've done some testing beyond that limit (you have, haven't you? Right? RIGHT? :), but since I've not looked through all the sources I can never be sure that at some point I'll say "I'd like to do..." and somebody will tell me "You can't, because internally Zopes this and that as...". That's how Windows tends to work. :-)
Alexander Staubo mailto:alex@mop.no http://www.mop.no/~alex/
_______________________________________________ Zope maillist - Zope@zope.org http://www.zope.org/mailman/listinfo/zope
(For developer-specific issues, use the companion list, zope-dev@zope.org - http://www.zope.org/mailman/listinfo/zope-dev )
-- Howard Clinton Shaw III - Grum St. Thomas High School #include "disclaimer.h"
At the risk of putting out a "me too" message. I would like to echo Howard's sentiments here. I think DC (all y'all) are doing an admirable job of managing the products, service and the community. Mui bueno. Jimmie Houchin At 03:14 PM 5/26/99 -0500, Howard Clinton Shaw III wrote:
On Wed, 26 May 1999, Alexander Staubo wrote:
Paul Everitt wrote: [Big Snip]
Paul, DC, let me just say once again, that in all the reading of articles and discussions and responses, never have I seen a negative image of Zope or DC go by. You guys are on target in maintaining the proper relationship with the community you have fostered and built around Zope. Keep up the excellent work.
-- Howard Clinton Shaw III - Grum St. Thomas High School #include "disclaimer.h"
On Wed, 26 May 1999, Alexander Staubo wrote:
The risk from my own point of view is really that Zope's design suddenly one day might prove inefficient. For example, what if Zope doesn't scale beyond 1000 objects? That's my fear, because such a break in scalability may be deep in its design. I doubt that Zope doesn't handle 1000 objects, because obviously you've done some testing beyond that limit (you have, haven't you? Right? RIGHT? :), but since I've not looked through all the sources I can never be sure that at some point I'll say "I'd like to do..." and somebody will tell me "You can't, because internally Zopes this and that as...". That's how Windows tends to work. :-)
Please, mentioning Zope and Windows in the same sentence is very unfair. Just try to start poking through Windows' source code next time you find an issue with it. :) Seriously though, as Paul points out, you are not solely at the mercy of Digital Creations if there is a bug or even an internal design issue. You can solve it yourself or hire someone to fix it, though I am certain that DC would be very interested to hear of any bugs in Zope. Point is, they've given up the source which is giving you, the customer, ultimate control over the program. One thing that might help alleviate concerns about scalability is having a page on the site that details some hard numbers about real-world Zope installations (ie. # of objects stored, numbers of access per day handled, size in megabytes of the database, etc). Just my 2 cents. ------- Jordan B. Baker -- jbb@spyderlab.com weaving the web @ http://www.spyderlab.com
On Wed, 26 May 1999, Alexander Staubo wrote:
It isn't just an issue of compatibility, but also of speed, stability, and scalability. Being able to have the Zope piggyback an MSSQL, DB2, or Oracle server is definitely a huge comfort, because a _lot_ of man hours -- arguably more than in the case of Zope -- have been put into those RDBMS tweaking them and securing them. Instead of spending lots of Yes. But Zope does need an object oriented hierarchical database. Storing objects in one or more SQL tables buys you the worst of both worlds: -) You still do have ``binary garbage'' in effect. -) A extremly good relational table management, when in fact you need hierarchical data management, may be beaten in performance, etc. by a average to good hierarchical implementation. -) You pay the SQL server, etc. -) I'm not that much convinced that SQL used in this manner "SELECT data FROM TABLE WHERE pos=123456;" is much faster than "f.seek(123456); f.read(512)"
I can understand your concerns, as they mirror my own very strongly. (Just lookup my rants in the mailing list archive :) ) I've returned to Zope because: -) My current projects don't depend upon thread level concurrence. And Z2 is coming, where exactly this problem is being addressed. -) The ZPL has become compatible with our mindset :), so perhaps it's better to work with the community, instead of playing Rambo ;) -) I've found some mental balance on this object base stuff. Really important data goes into a SQL server. The webinterface goes to Zopebase.
time finding new, esoteric ways of optimizing Zope's DB, you can effectively delegate that work to people who have spend their whole lives with their heads inside database algorithms. (Although I admit that B-trees are nice.) But the problem as pointed out is, that SQL server do not solve exactly the problem Zope Base solves. They solve a similiar problem, but that's possibly not enough.
scale and compatibility issues, but most of the benefits of using Zope would be mitigated if I ended up having to write SQL queries
Correct. Not really. Using pluggable Brains (==Python classes for the return set), you can have many of the benefits, and still have your data stored in the SQL server.
I don't agree (if the above sentence means that having write SQL queries will diminish the benefits of using Zope). SQL Methods greatly reduce the gap between the RDMS and Zope. It doesn't magically objectify the "relational" bits, but at least integrates them pretty well into the Zope object space. Ooops, exactly.
Creations publish products that are commercial and cost $, but I'd object to any Microsoft-ish pricing (eg., if you'd sold Confera for $1000 I'd definitely balk). So please be lenient, huh? But more than that I'd like to ask you what happens when you demote a commercial product to a free one. Isn't this unfair to all those people who might have bought it? If there is a need for something, than somebody will implement it. If it is needed only for a minority of customers, it might succeed some time without being ZPLed, but if there is a wide need for something, somebody will recode it to be free. Many things whose license is to restrictive inspire better free replacements. (Don't ask me why, but license insecurity creates an itch for many people.)
The risk from my own point of view is really that Zope's design suddenly one day might prove inefficient. For example, what if Zope doesn't scale beyond 1000 objects? That's my fear, because such a break in scalability may be deep in its design. I doubt that Zope doesn't handle 1000 objects, because obviously you've done some testing beyond that limit (you have, haven't you? Right? RIGHT? :), but since I've not looked through all the sources I can never be sure that at some point I'll say "I'd like to do..." and somebody will tell me "You can't, because internally Zopes this and that as...". That's how Windows tends to work. Then you fix it. I know design bugs are the most difficult to fix, being followed by ``nice-design-bad-performance'' bugs on the list; but the ZPL puts even that in your hand.
In my personal experience, Zope is being limited by: -) ZODB2 single threadness. Ok, it's less than fun to have to write a multithreaded object base, but ZODB3 will be this, so the problem will be solved shortly. -) Python. Well Python is really perfect, with perhaps the exception of performance. But again, this is a recognized problem, Python 1.3 was a desaster, 1.4 was bad, 1.5 is quite acceptable. And for the really problematic stuff, there is always our friend gcc on standby. Andreas -- Win95: n., A huge annoying boot virus that causes random spontaneous system crashes, usually just before saving a massive project. Easily cured by UNIX. See also MS-DOS, IBM-DOS, DR-DOS, Win 3.x, Win98.
participants (7)
-
Alexander Staubo -
Andreas Kostyrka -
Howard Clinton Shaw III -
Itamar S.-T. -
Jimmie Houchin -
Jordan B. Baker -
Paul Everitt