[Zope] zope performance issues and it just doesn't scale

[Matt Hamilton]

Trevor Warren wrote:

>  Bottomline....it just doesn't scale.  A simple 3 page
> test give me throughput so bad i can't speak of. Zope
> didn't scale...hence i tried Zeo/Zope....that didn't
> scale too(On the same machine). Am trying Zope/Zeo
> across machines now and will let you know about the
> results. Apache+Zope is something i havent looked at
> yet.

You must be doing *something* wrong.  Not sure what yet.  There are 
already quite a few large sites running Zope, and they seem to be doing 
just fine.

Our main Zope server is a single box (Dual PIII 1.4GhZ) and during the 
day runs at 10 req/sec averaged over a 5 minute period.  Not exactly 
huge, but this is driving over 60 *actual* sites, most with quite a bit 
of complexity (or rather not much time spent on optimization) and the 5 
minute load average never goes above 0.4.  And that is a server running 
a number of zope instances, databases, and other web related stuff.

www.lastminute.com is running Zope.  I am not sure of the specifics of 
hardware, but it is a ZEO cluster with squid caching.  I would imagine 
they load test to quite high loads.

We are working with a company to put a divisional 15,000 user intranet 
in place on a cluster of four ZEO (dual proc Xeon) clients and a single 
ZEO server.  This hardware is overkill for that load, but we want 
expansion capacity.  There will potentiallly be quite a bit of per-user 
personalisation, which is expensive.

I'm sure others have plenty of other case studies.  Chris McDonough has 
done quite a bit of study into this.  Trust him. :)

>  Probably python is the issue here...but accepting the
> facts will let us move ahead and improve chris. We
> can't sit on our laurels when there literally isn't

Yes, python could be an issue.  As I mentioned before, we saw issues 
with python threading on Solaris, and due to thread deadlocks the 
performance suffered very badly.  What OS are you running this on?  What 
database adapter are you using?  I rememeber there were some issues with 
some of the database adapters causing trouble with threading (again on 
Solaris) are you still actually calling the SQL method in your 'short 
circuited' tests?

Just checking our setup, and:

ab -n 50 -c 5 http://www.netsight.co.uk/  gives 22 req/sec
ab -n 100 -c 20 http://www.netsight.co.uk/  gives 30 req/sec

This is on a page that does a resonable amount of computation, it 
gathers news articles, truncates the summary, displays them, builds the 
menu dynamically etc.  I have not done any tuning on it, it was written 
in half an hour, and those results aboce are on a server that is already 
serving about 6 req/sec to real users.  The Zope server is behind 
Apache, but apache is not caching this page.

-Matt


-- 
Matt Hamilton                                       matth at netsight.co.uk
Netsight Internet Solutions, Ltd.        Business Vision on the Internet
http://www.netsight.co.uk                             +44 (0)117 9090901
Web Design | Zope/Plone Development & Consulting | Co-location | Hosting