On 2/14/07, Philipp von Weitershausen <philipp@weitershausen.de> wrote:
There will definitely be a Zope 2.11 and at this point I see no reason why there shouldn't be a Zope 2.12. We (the Zope Community, not necessarily Zope Corporation) will maintain Zope 2 as long as it's necessary. "Maintaining" in this case also means integrating more Zope 3 technologies with each release. Nowadays (Zope 2.10) you can already write applications, or at least base libraries, that work on both Zope 3 and Zope 2.
This is very helpful for setting a planning horizon, thank you.
I'm surprised you're experiencing poor scaling. I think Zope can scale pretty well, or at least it can be *made* to scale pretty well. It all depends on your setup and, of course, on your application.
I can't swear to the accuracy of what I'm about to say but it is an operating hypothesis that describes accurately behavior that we are seeing. Our application is highly dynamic, by which I mean that satisfying any given request may require a great deal of database activity. The database in question is almost always Oracle; I'm not counting the ZODB accesses, here. Oracle will handle a great many concurrent requests fairly gracefully, but its average transaction time is not always excellent, especially for moderately complicated queries. While most queries are sub-second, populating a page that requires multiple queries to fill in all its data fields, or even populating a page with only one or two complicated queries, can require greater than a single second of database I/O. This cost is offset by the ability to have other threads processing while your first thread waits for the database to come back. Zope likes to have 4 threads processing requests. This preference is documented in several places, and experiments have historically borne out the "4 thread hypothesis". We have been unable to realize significant performance improvements by increasing the number of available worker threads, but we are able to realize performance improvements by pointing two Zope servers against the same database. Zope isn't using much CPU, and it's not using much memory, but it peaks out at 4 concurrent requests; everything else goes in a queue behind them. If those 4 concurrent requests happen to take 4 or 5 seconds each, the queue builds deep behind them, possibly with very simple one-offs. Historically, we've addressed this by sticking another Zope out there. We have plans to run multiple Zope instances on a single machine, like they do (did?) at plone.org; this still requires configuration management work and testing. But the feeling is that we have a lot more hardware than we really should need, to handle the volumes we're dealing with, and we have that hardware because we need more "channels" to handle our concurrent traffic load.
As for source control, I figure all of your code (DTML, yuck) is in the ZODB. This went out of fashion a long time ago, most serious development happens on the filesystem (in Python packages) which can obviously be source-controlled very well.
A good deal of the code is in the ZODB, because it's (yuck) DTML documents, yes. We have another sizable chunk that lives on the filesystem. The challenge in managing the build/deploy process has been trying to find a way to keep those two conjoined. I suspect that any dependency on the ZODB at all is likely to be considered an impediment, and my quick eyeballing says it's not gone or even really optional under Zope 3 (but I'm sure we could work something out.)
Regarding "oh you'll hafta start over", it's pretty much true, if you want to switch to Zope 3. But nobody says you have to. You can do it incrementally by porting some of your app's functionality to Zope 3 components step by step (as mentioned already, most work in Zope 2). Big projects like Plone aren't rewriting their whole codebase either, but new development is completely Zope3-based. Their target platform still is Zope 2, though.
"You'll hafta start over" is only ever true in degrees, though, and I'm still trying to figure out what the degrees are. The business logic will still be the same. Can we hack together a DTML processor that allows us to export the DTML documents to the filesystem and publish them from there? Maybe, I don't know. Did someone else already do that? Don't know that either. How dramatically will our products need to change? Probably 75% of our Python code is written in a bastardized form of ExternalMethod; we might be able to leverage that unfortunate architectural choice to significant advantage during our porting phase. These are the kinds of questions I have, and I think the answers probably aren't easy or someone would have offered them up. Someone's going to need to learn enough Zope 3 to answer the questions. I'm not sure it will be me, but maybe. :) Thanks for all the insight, I'm making plans and pleas and I have a direction for my research now. :) Thanks, --G.