I will have to concur that Philip's book is a very good read. In addition to his book he has great suggestions for your library of other books. There are many of his ideas I think are wonderful, however I do have some philosophical disagreements on others. Some of his views of computers and programming are somewhat cynical even if based somewhat on fact. His version of scalability of which he focuses on a lot is basically throw lots of money at it. Start off with a highly scalable, multi-threaded, high quality webserver with an embedded scripting interpreter. In his case AOLserver with Tcl. He openly hates Tcl, but doesn't work towards using the language he likes where he works. Further more he thinks doing so is folly. (my interpretation) He states relational dbms are the only way to store data. Other technologies aren't as mature or capable. And on relational dbs only Oracle is sufficient. With others you are taking unnecessary risk. So to scale with Philip you develop with AOLserver/Tcl/ACS, buy big, expensive hardware, buy big expensive database. ie: throw lots of money. Since he prefers everything to be in a single machine you can only scale up processors so much before the $$$ start escalating. Price an 8 processor anything versus a Linux cluster of 8 machines. This is not enabling for the little guy. Yes I know of the ACS port to PostgreSQL. I think that's great. However it is going to be a step behind ArsDigita/ACS because it does require porting to become PostgreSQL able. He has disparaging remarks about app servers, many of which are true. However, I do not necessarily believe that learning Zope/webserver is more difficult than learning AOLserver/ACS. And when you've got Zope you've got Python instead of Tcl. For some of us, we like that. :) As a prodigal son here, I embraced Bobo/Zope early, became concerned about scalability, wandered the websphere, and have come back home. I have looked at Java/Servlets/Enhydra, AOLserver/ACS, etc. I decided to follow my heart. I like Python and Zope. I like the Python and Zope communities. This is where I want to make a difference and contribute. Zope can enable the idea person, the little guy with a twinkle in his eye and a big dream to pursue. Sure it may not "currently" scale as easily as AOLserver/ACS for big, busy sites. However, I think it is being worked on. I think that there are options within Zope, Digital Creations. To a very large extent Philip solves scalability with $$$dollars. With Zope you can still throw $$$dollars at Oracle. You can also throw some at Sybase, or whateverBase you desire. If that gains you scalability or the ability to sleep at night it's available. With ArsDigita/ACS the only supported option is Oracle. If I believed throwing $$$dollars to solve scalability was required. I would much rather throw $$$dollars at Digital Creations. Instead of a $20,000 to $500,000 on up license of Oracle which is per machine, per website, and potentially expires. Instead of requiring buying big, big hardware for $$$dollars. Throw the money towards Digital Creations or other Zope or Python developers to solve the issues which hold Zope/Python back. In the end you will have helped build a bigger, better open source software solution. As a little guy with ideas and dreams, this is my view. By the time a website grows enough to run into scalability and performance problems with Zope it could potentially be making enough $$$dollars to contribute to solutions. Try that with AOLserver/ACS/Oracle, start with $$$dollars and then hope for the best. My other thoughts wandered into employment. When I master either Zope or AOLserver/ACS, would I rather work for ArsDigita (yes, they pay well) or Digital Creations, if an opportunity presented itself in the future. As I know both are growing companies. I like Digital Creations. :) On a side note, on the AOLserver mail list there is effort to embed Python and get Zope running on AOLserver. I think it would be great to see Zope/Python being multi-thread, multi-processor able on a high performance open source, multi-threaded webserver. It would be awesome to have an embedded Python interpreter in AOLserver without a global threadlock that worked with AOLserver's multi-threaded API running Zope. On another side. A website I just learned about does a performance index of many sites and gives a average amount of time to load pages. This could put into perspective Zopes performance from the real world, from your own website. http://www.keynote.com/measures/consumer/consumer40.html Just some opinions. Jimmie Houchin Robert OConnor wrote:

Hi Kent, You wrote:

...

Does anyone out there have opinions on Phil Greenspun's app server; http://www.arsdigita.com/index.adp and the ArsDigita Community System http://www.arsdigita.com/pages/toolkit/index.html I want to set up a niche portal and am looking at Zope and the above ACS

Phillip has written a great book that I highly recommend to ALL Zope users:

"Philip and Alex's Guide to Web Publishing" http://www.amazon.com/exec/obidos/ASIN/1558605347

...

From Amazon review: "....[It] is both broadly conceptual and deeply technical, and it assumes that the reader is willing to think seriously about the challenge of building a content site, a community site, or an e-commerce store before plunging in.

Although heavily Unix-oriented, it does not set out to proselytize a product, or even suggest that there is only one way to solve certain technical challenges. Rather, it encourages the reader to think about Web content and functionality as something designed to help visitors answer questions or do something useful. This may sound nebulous, but his observations about why Web sites go bad are illustrated with many well-chosen examples....."

Bob's comments:

Ok, this book is also "FREE" and fully on the web at: http://www.photo.net/wtr/thebook/ and has reader comments... The dead tree version is on glossy paper with many of Phillip's excellent photos and is the only tech book that would set well on a coffee table.

Both AOLserver/ACS and ZOPE are open source ...great.

One disadvantage I see with AOLserver/ACS is the high cost of Oracle used as a back end. Some in the ACS community have talked of using other back end databases (particularly databases that handle transactions and are open sourced) but I don't know of how far this has progressed in the past three months. Philip does make a good point about the database passing the "ACID" test which is "Atomicity, Consistency, Isolation and Durability" See: http://www.photo.net/wtr/thebook/databases-choosing.html I am not sure that the Zope object database meets the ACID test.

ArsDigita sells it services to large clients with big bucks (along with givaways to good community projects.) I believe Digital Creations also serves large clients.

Zope is "cleaner" than AOLserver/ACS because it uses python rather than TCL and is "Object" based.

AOLserver/ACS seems to have has more "features" that are working on production sites. Is Zope still weak in the e-commerce area? The Zope portal is "almost" here. See Philip's photo.net site for a working community system. ...

I have played with both products but have yet to do productive sites for paying clients with either product. I am interested in the comments of others both on or off list.

-Bob OConnor zope@rocnet.com

_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )

Thanks, It was never my intention to imply that throwing $$$dollars at DC was required to improve scalability. What I said was based on perception of such requirement. I'll admit "currently" Zope does not scale as well or as elegantly as I would like. A statement I was making was that "if" I (or anyone) "believed" that scalability == $$$dollars, I would rather give to DC than Oracle. I believe Larry is doing all right on his own. :) I find Oracle pricing obscene. For the kind of $dollars it would take to license Oracle I would rather see what DC could do with the additional employees on beefing up Zope/ZODBs concurrency/scalabilities. There is a thread on the WebDB forum about a group of people talking about putting together a co-op to buy an Oracle license based on Oracles "unit" pricing scheme for a moderately powered single processor Intel machine it was going to run about $20,000. You would have to renegotiate if your site got busy and you added a processor. Yuck! I believe Zope's scalability issues will be solved over time regardless of external $dollars. I believe more $dollars could speed up the process, however it will still take some time. Currently ZEO is an option but I don't know how well understood it is by the community. Scalability Now can still be a sticky wicket. Future things on Zope's side: 1. Moore's law. 2. Advances in Python and it's performance. a. Python becomes fully multi-threaded, multi-processorable. b. Stackless Python gets adopted and performance improves. c. Viper 3. Advances in Zope, it's architecture and performance. a. Sam Rushing works miracles on Medusa/Zserver using Stackless Python and Continuations. b. Zope does process pooling with each process on a different processor. ZEO? c. ZODB does multiple data.fs's based on user design based on url path directories allowing different Zope processes on different processors to handle their respective dir. It would be nice if some C/Python programming guru/genius would solve Python's global threadlock issues where Python would scale on multiprocessor machines. This would potentially render scalability a moot point. The geniuses at AOLserver rewrote the threading of the built-in Tcl interpreter so that it would be multi-threaded and work properly with their nice multi-threaded webserver. Tcl's standard threading I don't think (I could be wrong) is as nice as Python's. If I had Oracle kind of money to throw around I would like to see Python's thread issue resolved where Python becomes fully multi-processorable. That would transparently scale Zope and benefit other Python user's as well. It would enable Python/Zope to embed well and play nice in multi-threaded webservers like AOLserver and others. IIRC Apache 2.0 will be multi-threaded. If Python scaled on multi-processors even a little guy could put together a dual-celery machine a serve a nice site. I agree wholeheartedly with you that there should be a concerted coordinated effort between DC and the community to improve scalability by allowing people to throw hardware at it. That was the primary issue causing me to wander the websphere looking at options. I've learned some things in the search, but I always looked back at Zope, even tho' my wife liked Enhydra's logo, she thinks the otter is cute. Zope's current logo is much nicer than the Purple and Gold. :) If ZEO currently scales well, then DC needs some testimonials on their website. There definitely needs to be some documentation best ways to... As you said, there is a consistent recurring thread of is ZServer alone or ZServer behind Apache better. Should I use a RDBMS or ZODB. ... Some of us have no legacy issues. From scratch what's the best way to design... I agree the solving of Performance/Scalability and Documentation issues would definitely help convert lookers into believers. I am currently working my way through the docs and plan on helping there. While I was wandering, in my heart Zope was the project I wanted to see succeed. I decided to put my mind where my heart was. Possibly someday some of my money too. :) Some of your other concerns I can't address. However, one of the nice things about Zope is that it is written in Python. Python imminently supports playing and algorithm improvement. :) Jimmie Houchin Patrick Phalen wrote:

[Jimmie Houchin, on Sat, 04 Mar 2000]

:: As a prodigal son here, I embraced Bobo/Zope early, became concerned :: about scalability, wandered the websphere, and have come back home. I :: have looked at Java/Servlets/Enhydra, AOLserver/ACS, etc. I decided to :: follow my heart. I like Python and Zope. I like the Python and Zope :: communities. This is where I want to make a difference and contribute. <snip> :: If I believed throwing $$$dollars to solve scalability was required. I :: would much rather throw $$$dollars at Digital Creations. Instead of a :: $20,000 to $500,000 on up license of Oracle which is per machine, per :: website, and potentially expires. Instead of requiring buying big, big :: hardware for $$$dollars. Throw the money towards Digital Creations or :: other Zope or Python developers to solve the issues which hold :: Zope/Python back. In the end you will have helped build a bigger, better :: open source software solution.

Welcome back, Jimmie. I enjoyed reading your analysis and agree with the thrust of it.

One thing you express, and which seems to have ossified into the current received wisdom, is that scalability will be improved when some mythical deep pockets consulting client pays to get it improved ; then everyone else will benefit. The problem is with the leap of faith it requires from the client.

I currently have several clients who are very interested in Zope. Their web sites already receive megahits. They are willing to pay money for consulting, but certainly won't bank their businesses on the chance that a magic key to scalability is found. As a consequence, the greatest commitment they will make is to install Zope on a Linux server on a VPN and "see how it goes" in a toy setting.

Therefore, I can testify that right now, today, Zope is being hurt by the lack of a story on scalability.

If economics or policy dictates that Zope requires an "Angel" to bankroll scalability research, we all have to be prepared for a scenario in which that Angel never appears. Personally, I don't like taking such a passive approach.

Two clear impediments to Zope's wide acceptance have been frequently discussed on this list -- documentation and performance.

The documentation "story" has travelled on an arc which might be expressed like this:

* Zope is Open Source. The community should document. * The community can't take on the deeper Zen issues * Volunteers from the community are time constrained * DC hires a technical writer * The ZDP is formed * ZDP suffers from the lack of time available to its volunteers * DC commits Amos to attack the general documentation problem and to evangelize documentation

Documentation currently seems headed on a good track. What have we learned from this trajectory? We've all learned that the Zope experiment is different from other Open Source projects. It *needs* the cooperation of *both* the Zope community and Zope's authors "shooting at the same basket" to solve this.

I believe scalability requires a similar coordinated attack and that it can't wait to be bankrolled.

ZEO, although a bit of a black box to us at the moment, sounds, on paper, like a logical and credible attack on one aspect of scalability -- the "throw hardware at it" approach.

No doubt there are other hidden opportunities. What kinds of hidden opportunities? Who knows; they're hidden. ;>) But can we bring them out of hiding?

Some of them are simple applications of available compute science, yet I don't see these codified or set down anywhere. Perhaps we should have a single HOW-TO repository for techniques and metrics people have discovered, or just happen to know, in the performance arena. This would be a FAQ along the lines of "I want to build my Zope site for best performance. What things should I do? What things should I avoid?" God knows this question comes up often enough.

Then there's the design and makeup of the Zope software itself. Its original design requirement probably didn't put scalability at the top of the stack. Here, there are probably other areas where the user community can't know enough about inner workings and overall design to contribute effectively -- but which require attention and a commitment from the folks at DC. I'll mention a couple of things that pop into my head. These are simply conjectural -- dialog starters. I'll be happy to be shown that I'm dead wrong about any of these conjectures.

Zope began as a brilliant notion, of Jim Fulton's devising. It began as a fews lines of code and has evolved to include better than two megabytes of code. DC has evolved from a company with one employee to a company with maybe a dozen engineers contributing to the code base and is looking to hire twenty more engineers.

Among this team of top notch talent, who takes ultimate responsibility for evangelizing and improving performance? Who leads the effort at code review, with an eye to marshalling refactoring efforts?

What would be an example? OK, Devil's Advocate: Here's one area of concern I have, perhaps unfounded. Digital Creations utilizes Use Cases as a tool for design, in the Ivor Jacobson tradition. Use Cases have many proponents and some detractors. One detractor is Bertrand Meyer, author of the classic _Object-Oriented Software Construction_. His beef with the use of Use Cases is that it poses a fundamental risk: it encourages a functional approach, based on processes (actions). He sees this as the reverse of O-O decomposition, which focuses on data abstraction. He believes it carries a risk of reverting, under the heading of object-oriented development, to the traditional forms of functional design, by considering what the system does, rather than what it does it to. He believes that, due to this risk, Use Cases should be confined to use as a validation tool, not a design tool, to avoid closing interfaces prematurely.

So let's say that Use Cases is a useful tool, but one that has a subtle built-in gotcha. What are the dangers? Naturally, one is negative impact on code maintainability/reusability.

Another may arise due to the need to balance two competing technical concerns: the desire to encapsulate abstractions, and the need to make certain abstractions visible to other modules. With regard to the dynamics of subprogram calls, the placement of declarations within modules can greatly affect the locality of reference and thus the paging behavior of a virtual memory system. Poor locality happens when subprogram calls occur across segments and lead to cache misses and page thrashing, which can ultimately slow down the whole system.

Is Zope prey to this? I don't have a clue. Does anyone? Like I say, I'd be happy to be shown that this concern is unfounded. But this is the sort of creeping problem which can enter into a growing code base like Zope's, as it is worked on by numerous people, each devoted on one module or interface. Combatting such problems probably requires having someone (someone blessed with both Jim Fulton's dedication to O-O principles and an unbending zealotry for absolute performance) committed to periodically flying above the code at a sufficient distance to see such patterns and making bombing runs against the target problems. As busy as everyone at DC is, I'm not certain that anyone is doing that at the moment.

_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )