RE: [Zope] ZOPE, Medusa and the ZD WebBench 3.0 test suite...
Lies, damn lies and benchmarks. -Samuel Clements via me.
My question, and the one I expect is on many of your minds, is... How does a ZOPE/Medusa server config stack up? My hope and expectation that the underlying Medusa architecture might really shine when compared to the pre-allocating multi-process approach of Apache, etc.
Well, this is for lack of a better statement, irrelevent to Zope. They are asking (last I looked) for a static-page system, with some tiny CGIs, but this isn't really what Zope is about. EVERY page is Zope is dynamic, so if you compared a 100% CGI site with Zope, then it *might* be a valid comparison, if you also pulled all the data out of a database on the fly as well... IMPORTANT: The only benchmark that matters is the one that is your application, and on that, Zope excells. As for asyncronous v. threading/forking servers. Zeus, which is the fastest 'static' web server int he world, and what everyone uses for benchmarking on UNIX is also an asyncronous server. Different servers have different behaviours. One of the common problems with IIS v. Apache/etc is that Apache isn't tuned to saturate a small number of clients (which is all they are simulating) with very big pipes, but instead, a huge number of clients with smaller bandwidth (what the Internet really looks like). Also, IIS4 runs a "cache" inside the server, which you could simulate iwth Squid, or some such... One must be careful comparing the flavor of two things sold as oranges when one is distinctly shaped like a pumpkin. Chris -- | Christopher Petrilli Digital Creations | petrilli@digicool.com http://www.digicool.com
Thanks for the thoughtful response, Chris, but something to add to the mix is that this review marks the debut of ZD's WebBench 3.0. According to PC Mag this version addresses the static, dynamic, and secure configs in a more realistic manner than earlier versions of the test. (There may be info about this test suite on the ZDNet site (there is a download dated 4-6-99 in the software library), if not and you don't have access to the PCMag issue at hand, drop me a note and I can scan and e-mail them to you, Chris, so you can see if the updated benchmark addresses your concerns.) So, assuming that they may have a more realistic "spread" of typical serving demands, the question comes back to "Can a ZOPE/Medusa config be run against WebBench 3.0?" Despite whatever we and Twain may think about benchmarks in general, from a marketing/sales standpoint saying "We don't believe in benchmarks and none are appropriate for our technology," does not go over very well with potential client/customers. (Nor do non-standard tests go over well since they don't allow "comparative shopping.") My experience is that it is much better to at least subject yourself to such "standard benchmarks" and have a compelling "footnotes and explanations" to address any mismatch between your technology and an industry benchmark. A curious ZOPEer, --Jim Salmons-- <salmons@sohodojo.com> -----Original Message----- From: Chris Petrilli [mailto:petrilli@digicool.com] Sent: Monday, May 24, 1999 11:31 AM To: 'Jim Salmons'; zope@zope.org Subject: RE: [Zope] ZOPE, Medusa and the ZD WebBench 3.0 test suite... Lies, damn lies and benchmarks. -Samuel Clements via me.
My question, and the one I expect is on many of your minds, is... How does a ZOPE/Medusa server config stack up? My hope and expectation that the underlying Medusa architecture might really shine when compared to the pre-allocating multi-process approach of Apache, etc.
Well, this is for lack of a better statement, irrelevent to Zope. They are asking (last I looked) for a static-page system, with some tiny CGIs, but this isn't really what Zope is about. EVERY page is Zope is dynamic, so if you compared a 100% CGI site with Zope, then it *might* be a valid comparison, if you also pulled all the data out of a database on the fly as well... IMPORTANT: The only benchmark that matters is the one that is your application, and on that, Zope excells. As for asyncronous v. threading/forking servers. Zeus, which is the fastest 'static' web server int he world, and what everyone uses for benchmarking on UNIX is also an asyncronous server. Different servers have different behaviours. One of the common problems with IIS v. Apache/etc is that Apache isn't tuned to saturate a small number of clients (which is all they are simulating) with very big pipes, but instead, a huge number of clients with smaller bandwidth (what the Internet really looks like). Also, IIS4 runs a "cache" inside the server, which you could simulate iwth Squid, or some such... One must be careful comparing the flavor of two things sold as oranges when one is distinctly shaped like a pumpkin. Chris -- | Christopher Petrilli Digital Creations | petrilli@digicool.com http://www.digicool.com
On Mon, 24 May 1999, Jim Salmons wrote:
Despite whatever we and Twain may think about benchmarks in general, from a marketing/sales standpoint saying "We don't believe in benchmarks and none are appropriate for our technology," does not go over very well with potential client/customers. (Nor do non-standard tests go over well since they don't allow "comparative shopping.")
My experience is that it is much better to at least subject yourself to such "standard benchmarks" and have a compelling "footnotes and explanations" to address any mismatch between your technology and an industry benchmark.
I would agree with Chris that benchmarks are generally biased but you are right they are essential for marketing purposes. So why not give them marketing non-sense ;-) I mean run medusa, just pure medusa no Zope to serve static html pages, and even better cache the pages in memory. It is Zope's web server after all. Talking nonsense Pavlos
participants (3)
-
Chris Petrilli -
Jim Salmons -
Pavlos Christoforou