[Zope] Zope Scalability

[John Snowdon]

>--On 5. Oktober 2005 17:37:08 +0100 Tom Wilde 
><tom.wilde at odconsultancy.com> 
>wrote:
>
>> Hi folks,
>>
>> I've been using Zope for a while now and found it to be an excellent
>> solution however I've got a few enterprise questions about the
>> scalability of Zope that just need to be answered.
>>
>> [ nb: I'm using V2.72 at the moment, not having moved over to V3 ]
>>
>> ExtFile
>> Managed to serve 500,000 files through extfile - anybody 
>come across an
>> upper limit here? Planning to serve larger sites off a SAN 
>hopefully -
>> has anyone tried this?
>
>no idea

For static content we still use Apache.

>>
>> Zeo and Cataloging
>> If I have my site on one server and ZCatalog (using TextIndexNG) on
>> another server (connected by ZEO) where does the actual 
>index operation
>> occur? Do just the results get propogated to the catalog?
>
>The indexing work is always happens on the ZEO clients, not on 
>the server.
>Just the results are transferred to the ZEO Server.
>
>>
>> LDAP Authentication
>> I understand that can use the LDAPUserFolder Product to provide
>> authentication against an ldap server. Could I throw,say, 
>50000+ users
>> at zope using this system?
>
>This should work in general. Possibly it requires some 
>tweaking here and 
>there depending on your usecases.

We use LDAPUserFolder as the backend authentication store for our online
learning environment here at newcastle. It copes rather well with
several thousand students and many groups - I'd imagine it will scale as
well as your LDAP setup can. (Our LDAP server is a pretty low-end Sun
V60 - single Xeon 2.8GHz with 2Gb of ram and is very rarely under any
significant load).

>>
>> Load Balancing and Cookie Based Sessions
>> Currently we use cookie based sessions via cookie crumbler 
>presumably in
>> a load balanced server setup we'd have to host acl_users ona  ZEO'd
>> izope instance somewhere on the backend?
>
>
>I don't see what sessions have to do with acl_users. In your 
>ZEO setup you 
>have identical data except sessions on every ZEO client. You 
>just have to 
>ensure somehow that users get to the same ZEO client for the 
>lifetime of 
>the session. How this is done depends basically on the load balancer.

We initially tied users sessions to individual ZEO clients (6 x dual
Opteron nodes) and it worked well. However we have since migrated the
session information to be served via the ZEO server in the same manner
as the data.fs; we now get the benefit of much finer grained load
balancing and are very pleased with the performance we get. As a side
effect, it means we can host multiple ZEO clients from different systems
on one box without fear of a particular ZEO client bringing the box to
it's knees.

As an aside, we find management of ZEO clients much easier if each ZEO
client of a particular system shares the same products and external
methods via an NFS share. That way we can untar one product and push it
out to all of the clients simultaneously.

-John
--------------
Faculty of Medical Sciences Computing
University of Newcastle upon Tyne