Toby Dickenson wrote:
On Fri, 14 Jul 2000 14:12:33 -0600, Bill Anderson <bill@libc.org> wrote:
Roman Milner wrote:
I'm wondering if anyone can suggest something good to run in front of 2 zopes talking to a zeo server - for failover and load balancing. I have been using a tool called 'balance' which works really well - but for failover it only works if the socket to one of the zope servers doesn't connect.
We occasionally have a situation where the socket will still answer but the zope server hanges and never returns an answer. So, I'm looking for something with configurable timeouts on responses. Does any one know of anything like that?
You didn't specify the platform, so I'll suggest a nice, cost-effective one: Linux with IPVS (IP Virtual Server). www.linux-ha.com has more information.
That looks good on paper, and I may be setting up a trial system next month.
I'm doing it this week, I'll keep you posted ;-)
One disadvantage is that solution is that each Zope will have poor locality-of-reference within the object database. I think I can avoid that using a squid redirector (www.squid-cache.org). Ill post any news.
What do you mean? -- Do not meddle in the affairs of sysadmins, for they are easy to annoy, and have the root password.
On Mon, 17 Jul 2000 05:45:49 -0600, Bill Anderson <bill@libc.org> wrote:
I'm wondering if anyone can suggest something good to run in front of 2 zopes talking to a zeo server - for failover and load balancing. I
One disadvantage is that solution is that each Zope will have poor locality-of-reference within the object database. I think I can avoid that using a squid redirector (www.squid-cache.org). Ill post any news.
What do you mean?
Suppose your zope site www.contrived-example.com is comprised of many largely independant sections, http://www.contrived-example.com/section1, http://www.contrived-example.com/section2 etc. Your multiple Zopes can all serve all of these sections, however theres not enough storage for each machine to hold all the sections simultaneously. You can make better use of ZODB's in-memory cache and the ZEO pickle cache if the requests for /section1 usually go to the same server. However, you don't want to hardwire this relationship since any other machine should handle /section1 if its 'home' machine goes down, or is busy. This problem can't be solved without parsing http headers, so low-level solutions such as www.linux-ha.com are not a total solution. Toby Dickenson tdickenson@geminidataloggers.com
Toby Dickenson wrote:
On Mon, 17 Jul 2000 05:45:49 -0600, Bill Anderson <bill@libc.org> wrote:
I'm wondering if anyone can suggest something good to run in front of 2 zopes talking to a zeo server - for failover and load balancing. I
One disadvantage is that solution is that each Zope will have poor locality-of-reference within the object database. I think I can avoid that using a squid redirector (www.squid-cache.org). Ill post any news.
What do you mean?
Suppose your zope site www.contrived-example.com is comprised of many largely independant sections, http://www.contrived-example.com/section1, http://www.contrived-example.com/section2 etc.
Your multiple Zopes can all serve all of these sections, however theres not enough storage for each machine to hold all the sections simultaneously.
As I understand ZEO, each machine _doesn't_ hold the site. The ZEO clients (servers) communicate with a central ZSS (Zope Storage Server). So in this contrived example, the problem is non-extant. ;-)
You can make better use of ZODB's in-memory cache and the ZEO pickle cache if the requests for /section1 usually go to the same server.
AIUI (I'm no ZEO expert, I just use it ;) ), you can have each server cache a certain amount, thus ameliorating the problem somewhat. -- Do not meddle in the affairs of sysadmins, for they are easy to annoy, and have the root password.
On Tue, 18 Jul 2000, Bill Anderson wrote:
Your multiple Zopes can all serve all of these sections, however theres not enough storage for each machine to hold all the sections simultaneously.
As I understand ZEO, each machine _doesn't_ hold the site. The ZEO clients (servers) communicate with a central ZSS (Zope Storage Server). So in this contrived example, the problem is non-extant. ;-)
You can make better use of ZODB's in-memory cache and the ZEO pickle cache if the requests for /section1 usually go to the same server.
AIUI (I'm no ZEO expert, I just use it ;) ), you can have each server cache a certain amount, thus ameliorating the problem somewhat.
Yes, however his point is that by having each Zope instance 'predominantly' serving one portion of the site, its cache will contain more objects relevant, and thus be just that little bit faster. Personally, I find this such a simple idea that it MUST be good. (o8 So much so, in fact, that I've decided to have a crack at writing just such a redirector. I feel the Zope world (and others, most likely) could benefit from a 'preferential' redirector. Watch this space. (o8 Have a better one, Curtis Maloney.
Curtis Maloney wrote:
On Tue, 18 Jul 2000, Bill Anderson wrote:
Your multiple Zopes can all serve all of these sections, however theres not enough storage for each machine to hold all the sections simultaneously.
As I understand ZEO, each machine _doesn't_ hold the site. The ZEO clients (servers) communicate with a central ZSS (Zope Storage Server). So in this contrived example, the problem is non-extant. ;-)
You can make better use of ZODB's in-memory cache and the ZEO pickle cache if the requests for /section1 usually go to the same server.
AIUI (I'm no ZEO expert, I just use it ;) ), you can have each server cache a certain amount, thus ameliorating the problem somewhat.
Yes, however his point is that by having each Zope instance 'predominantly' serving one portion of the site, its cache will contain more objects relevant, and thus be just that little bit faster.
Which is why I said 'somewhat' ;-) Now, depending on if you set a shared FS across the systems, this cache could theoretically be shared. If for example, you usef afs/codafs, you could have each server using the same FS, reading the same cache, unless something in ZEO prevents two processes form using the cache, of course. Now, AFS/Codafs are not for the faint of heart, but I get the feeling that someone willing to try an IPVS setup is likely made of sturdier stuff. ;-)
Personally, I find this such a simple idea that it MUST be good. (o8 So much so, in fact, that I've decided to have a crack at writing just such a redirector. I feel the Zope world (and others, most likely) could benefit from a 'preferential' redirector.
Roxen can do this to some extent. Not sure how well, since I am not using that aspect of it, but I do see it in the modules. I do know thet eddieware can also do this: """ The Eddie Intelligent HTTP Gateway package allows specialised functions to be allocated to specific Back End Servers. As each user request arrives, it is parsed by the Front End Server which then: o Splits multiple requests within HTTP 1.1 persistent connections into a number of individual requests. o Sends the individual requests to Back End This allows a system administrator to, for example, dedicate certain Back End Servers to be CGI processing engines, while other machines may be dedicated to acting as image repositories. The individual machines may then be tuned to optimise their performance to these specific tasks. """ I have considered trying out eddieware, but haven't yet. -- Do not meddle in the affairs of sysadmins, for they are easy to annoy, and have the root password.
Curtis Maloney wrote:
Yes, however his point is that by having each Zope instance 'predominantly' serving one portion of the site, its cache will contain more objects relevant, and thus be just that little bit faster.
Personally, I find this such a simple idea that it MUST be good. (o8 So much so, in fact, that I've decided to have a crack at writing just such a redirector. I feel the Zope world (and others, most likely) could benefit from a 'preferential' redirector.
The way I would do this is have section1.contrived-example.com section2.contrived-example.com section3.contrived-example.com with siteAccess, and then each zope would serve it according to it's IP (though each "could" serve each site). Then you can use whatever IP/DNS load balancing tool your heart desires. a thought, -- ethan mindlace fremen Zopatista Community Liason Abnegate I!
participants (4)
-
Bill Anderson -
Curtis Maloney -
ethan mindlace fremen -
Toby Dickenson