On Thu, 20 Jul 2000, Bill Anderson wrote:
Toby Dickenson wrote:
On Tue, 18 Jul 2000 16:08:48 -0600, Bill Anderson <bill@libc.org>
wrote:
I might be reading more into his words than was intended, but I think this demonstrates the problem. Distributing multiple requests for one section across multiple servers is (what I consider to be) undesirable.
You can actually do it either way. Curtis (AIUI) complained that the method described meant your site depended upon each of th esection's servers being up, that there was no redundancy. So I described a way of doing it with redundancy.
What you described doesn't scale up to having 1000's of sections (which I was assuming, and I think Curtis was too). If this isn't a problem, then your solution is great.
I don't understand why you think it doesn't. DNS has clearly demonstrated the ability to handle 'thousands', and the entire scalability of a cluster is the addition of machines. You appear to be desirous of having a machine handle a section. Thus, for thousands of sections, you have thousands of machines. Again, with a ZEO clusters the bottleneck/SPOF would be the ZSS, but that _could_ be worked aorund, and has nothing to do with 'sections' of a website.
Bill, Whilst the structures you've described are very effective, your example of libc.org required one thing in particular that I'm not sure is available: prior knowledge of which sections will be hit hardest. Essentially, your setup allows any 'server' to become a 'server cluster' for scaling purposes. Great! So, if for now on we assume 'server' can mean 'single or cluster of servers'.... The desire isn't for fixed server<->section relationship. Instead, a 'preference' for that section to go to a particular server, so that the request 'hopefully' goes the server with the greatest chance of having the relevant objects in cache. In fact, with the further information provided, what you really want is for requests from a particular client to go to the same server. This would be better served with a redirection to a server specific domain name (serverN.mysite.com). However, for the initial request, your best choice is to go to the server that last served those pages. Since dynamically tracking this info would be onerous, by encouraging requests for one section toward a particular server, you improve the chances of it holding the relevant objects in cache, with merely a fraction of the processing/data overheads.
Beyond that, your bottleneck would be networking. Whether yoour individual BE servers responded directly to the web browser, or whether they were channeled through a single/multiple FrontEnd servers. The decision to implement a BE->Client vs. a BE->FE->Client topology has not been discussed, as it is irrelevent to the discussion.
Ah, topology. (I'm leaving it there. I really don't have time to get into this fully :)
In fact, come to think of it, I have noticed many sites redirect a /foo/bar usr to a foo.domain.com or bar.domain.com.
EddieWare does do 'intellgient' caching
eddieware is on my list of option to try out next month... Ill keep you posted
Cool.
Have a better one, Curtis <dtml-var standard_work_disclaimer>