[Zope] ZEO and a front end...

[Bill Anderson]

Curtis Maloney wrote:
> 
> On Thu, 20 Jul 2000, Bill Anderson wrote:
> > Toby Dickenson wrote:
> > > On Tue, 18 Jul 2000 16:08:48 -0600, Bill Anderson <bill@libc.org>
> > >
> > > wrote:
> > > >> I might be reading more into his words than was intended, but I think
> > > >> this demonstrates the problem. Distributing multiple requests for one
> > > >> section across multiple servers is (what I consider to be)
> > > >> undesirable.
> > > >
> > > >You can actually do it either way. Curtis (AIUI) complained that the
> > > >method described meant your site depended upon each of th esection's
> > > >servers being up, that there was no redundancy. So I described a way of
> > > >doing it with redundancy.
> > >
> > > What you described doesn't scale up to having 1000's of sections
> > > (which I was assuming, and I think Curtis was too).  If this isn't a
> > > problem, then your solution is great.
> >
> > I don't understand why you think it doesn't. DNS has clearly
> > demonstrated the ability to handle 'thousands', and the entire
> > scalability of a cluster is the addition of machines. You appear to be
> > desirous of having a machine handle a section. Thus, for thousands of
> > sections, you have thousands of machines. Again, with a ZEO clusters the
> > bottleneck/SPOF would be the ZSS, but that _could_ be worked aorund, and
> > has nothing to do with 'sections' of a website.
> >
> Bill,
> 
>         Whilst the structures you've described are very effective, your example of
> libc.org required one thing in particular that I'm not sure is available:
> prior knowledge of which sections will be hit hardest.

You start with the most likely suspects, and then after a given time
interval, you adjust as needed. *most* site admins have a good idea of a
given section being more popular or frequented when the site is built.
That is as good a start as any other, if not better.

> 
>         Essentially, your setup allows any 'server' to become a 'server cluster' for
> scaling purposes.  Great!  So, if for now on we assume 'server' can mean
> 'single or cluster of servers'....

A logical assumption.
 
>         The desire isn't for fixed server<->section relationship.  Instead, a
> 'preference' for that section to go to a particular server, so that the
> request 'hopefully' goes the server with the greatest chance of having the
> relevant objects in cache.

I see that it may not have been clear, but my ecample provided just
that. A preference is indicated by the weight given to servers and
sections. Let us say I have three servers. Fo rthe whole site, two get a
weight of 2, whilst a third gets a weight of 1. This third one, however,
gets a weight of 2 for the members section, whilst the other two get a
weight or 1. This provides a preference for server3 to serve up the
members section, though it is not a direct-only mapping. how does this
not fit the 'hopefully' desire?

If you _wanted_ a direct-only, you simply remove servers 1 and 2 from
the list of the members section. The really neat thing about this is
that it can be done at runtime.
 
>         In fact, with the further information provided, what you really want is for
> requests from a particular client to go to the same server.  This would be
> better served with a redirection to a server specific domain name
> (serverN.mysite.com).  However, for the initial request, your best choice is
> to go to the server that last served those pages.
> 
>         Since dynamically tracking this info would be onerous, by encouraging
> requests for one section toward a particular server, you improve the chances
> of it holding the relevant objects in cache, with merely a fraction of the
> processing/data overheads.

Right. I agree that tracking all of this would be onerous, which is why
I said I don't think it is worth the effort, and would cost more than it
saved. The scenario I described gives a preference for sections to go to
a particular server, thus giving you the 'encouragement'. :^)


> 
> > Beyond that, your bottleneck would be networking. Whether yoour
> > individual BE servers responded directly to the web browser, or whether
> > they were channeled through a single/multiple FrontEnd servers. The
> > decision to implement a BE->Client vs. a BE->FE->Client topology has not
> > been discussed, as it is irrelevent to the discussion.
> 
> Ah, topology.  (I'm leaving it there.  I really don't have time to get into
> this fully :)

Yeah, topology is where the umm ... electrons hits the wire.
 
Mebbe I'll post this stuff to the Wiki ... the question is .,.. which
one?

--
Do not meddle in the affairs of sysadmins, for they are easy to annoy,
and have the root password.