[Zope] zope performance issues and it just doesn't scale

[Jake]

Just to throw into the debate:

AMD 2.13 Ghz, 1.5 GB Ram, 2 x 36 GB SCSI (Raid 1)

- Red Hat Enterprise Linux ES 2.1
- Python 2.3.3, Zope 2.7, Apache 2 Proxy Pass to Zope VHM
- Also runs mail, webmail, other apache sites, daily stats, etc

All the Zope sites combined run at about 100,000 hits a day (give or take
on an average day). Since I have a lot of RAM (Zope/Python love RAM) I
have it up to 40,000 objects per thread and about 10 threads active. It
runs at about 45% of RAM with 230,000 objects loaded out of 110,000+
objects in ZODB with Zope being up (no restart) for 33 days.

So all, of that combined, the server on a daily average (using sa command)
at about 3 to 5% of capacity.

Now, I have some RAM caches, etc but there are also some Zope apps that
both read/write to the ZODB a lot (AdManager) and some that take some
horsepower (ZUBB).

Jake

-- 
http://www.ZopeZone.com


Matt Hamilton said:
> Trevor Warren wrote:
>
>>  Bottomline....it just doesn't scale.  A simple 3 page
>> test give me throughput so bad i can't speak of. Zope
>> didn't scale...hence i tried Zeo/Zope....that didn't
>> scale too(On the same machine). Am trying Zope/Zeo
>> across machines now and will let you know about the
>> results. Apache+Zope is something i havent looked at
>> yet.
>
> You must be doing *something* wrong.  Not sure what yet.  There are
> already quite a few large sites running Zope, and they seem to be doing
> just fine.
>
> Our main Zope server is a single box (Dual PIII 1.4GhZ) and during the
> day runs at 10 req/sec averaged over a 5 minute period.  Not exactly
> huge, but this is driving over 60 *actual* sites, most with quite a bit
> of complexity (or rather not much time spent on optimization) and the 5
> minute load average never goes above 0.4.  And that is a server running
> a number of zope instances, databases, and other web related stuff.
>
> www.lastminute.com is running Zope.  I am not sure of the specifics of
> hardware, but it is a ZEO cluster with squid caching.  I would imagine
> they load test to quite high loads.
>
> We are working with a company to put a divisional 15,000 user intranet
> in place on a cluster of four ZEO (dual proc Xeon) clients and a single
> ZEO server.  This hardware is overkill for that load, but we want
> expansion capacity.  There will potentiallly be quite a bit of per-user
> personalisation, which is expensive.
>
> I'm sure others have plenty of other case studies.  Chris McDonough has
> done quite a bit of study into this.  Trust him. :)
>
>>  Probably python is the issue here...but accepting the
>> facts will let us move ahead and improve chris. We
>> can't sit on our laurels when there literally isn't
>
> Yes, python could be an issue.  As I mentioned before, we saw issues
> with python threading on Solaris, and due to thread deadlocks the
> performance suffered very badly.  What OS are you running this on?  What
> database adapter are you using?  I rememeber there were some issues with
> some of the database adapters causing trouble with threading (again on
> Solaris) are you still actually calling the SQL method in your 'short
> circuited' tests?
>
> Just checking our setup, and:
>
> ab -n 50 -c 5 http://www.netsight.co.uk/  gives 22 req/sec
> ab -n 100 -c 20 http://www.netsight.co.uk/  gives 30 req/sec
>
> This is on a page that does a resonable amount of computation, it
> gathers news articles, truncates the summary, displays them, builds the
> menu dynamically etc.  I have not done any tuning on it, it was written
> in half an hour, and those results aboce are on a server that is already
> serving about 6 req/sec to real users.  The Zope server is behind
> Apache, but apache is not caching this page.
>
> -Matt
>
>
> --
> Matt Hamilton                                       matth at netsight.co.uk
> Netsight Internet Solutions, Ltd.        Business Vision on the Internet
> http://www.netsight.co.uk                             +44 (0)117 9090901
> Web Design | Zope/Plone Development & Consulting | Co-location | Hosting
>
> _______________________________________________
> Zope maillist  -  Zope at zope.org
> http://mail.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists -
>  http://mail.zope.org/mailman/listinfo/zope-announce
>  http://mail.zope.org/mailman/listinfo/zope-dev )
>