[Zope-CMF] backup strategies

[Paul Winkler]

Thanks Sean, this one goes in the keeper file :)
Happily we have already got mon, heartbeat, and
failover working. 

--PW

On Fri, Feb 07, 2003 at 10:59:42AM -0800, sean.upton@uniontrib.com wrote:
> Cliff's notes version of ZSS HA clustering, assuming you have some sort of
> DirectoryStorage replication in place:
> 
> re: ZSS clustering.  You could use mon as well as heartbeat.  Heartbeat
> would be set up on a 2-node cluster of ZSS machines.  If the primary
> seriously died, then the backup would take over its IP address via
> gratuitous ARP.  
> 
> Heartbeat also manages resource with init-like scripts, when the takeover
> started, it would start up a ZSS process on a replicated DirectoryStorage
> after taking over the IP.  For safety, you would want to likely kill the
> primary server to keep it from replicating to the backup after the takeover.
> You could do this by using a power-device (STONITH: Shoot The Other Node In
> The Head).
> 
> The takeover that heartbeat performs could be initiated under 2 conditions:
> 
> - totally dead primary ZSS box.  Heartbeat does this when the primary stops
> sending heartbeats that the secondary can see.
> - Half-dead box: lack of ZSS service on tcp socket.  Use a mon alert script
> to trigger heartbeat takeover.
> 
> ZEO clients would retry connection after a timeout from a severed TCP
> connection to the ZSS, and see that it is back up.
> 
> Sean
> 
> -----Original Message-----
> From: Paul Winkler [mailto:pw_lists@slinkp.com]
> Sent: Friday, February 07, 2003 10:13 AM
> To: zope-cmf@zope.org
> Subject: Re: [Zope-CMF] backup strategies
> 
> 
> On Fri, Feb 07, 2003 at 04:09:17PM +0000, Sally Owens wrote:
> > Is there any way for example to *test* Data.fs when you back it up (to be 
> > sure that you are not backing up corrupt data)?
> 
> there's a utility in utilities/ZODBTools/fstest.py that checks
> for errors. Run a cron job that runs this tool and mails you the
> result.
> 
> There is also another utility, fscheck.py that gives more extensive
> reports, and IIRC is new to Zope 2.6.
> 
> > What backup strategies do other organisations employ to minimize the risk 
> > of your Zope/CMF sites falling over?
> 
> be sure not to overlook the advantage of running zope
> behind a proxy, especially if your system will ever be 
> exposed to the internet.  ZServer is not designed to handle
> DOS attacks, or very high traffic in general.  A proxy
> such as Squid can both mitigate this, and with proper
> caching greatly decrease the load on your Zope system.
> 
> As for protecting our data, currently much the same as you're suggesting.
> We make regular backups of Data.fs (using cp), and regularly
> run fstest.py.
> Strongly considering a move to DirectoryStorage on reiserfs.
> http://dirstorage.sourceforge.net/
> 
> This would enable, among other things, incremental backups and 
> semi-live replication using a custom shell script (and soon
> there will be a feature that supports live replication.)
> 
> 
> The cool thing about live replication is that you could in
> theory eliminate the single point of failure with Zope / ZEO: the ZEO
> server.
> By running monitoring software such as mon, you could
> set up a system that fails over from one ZEO cluster to another.
> Not trivial to set up, but I think it should work.
> 
> 
> -- 
> 
> Paul Winkler
> http://www.slinkp.com
> Look! Up in the sky! It's ANDREA SNOWBALL!
> (random hero from isometric.spaceninja.com)
> 
> _______________________________________________
> Zope-CMF maillist  -  Zope-CMF@zope.org
> http://mail.zope.org/mailman/listinfo/zope-cmf
> 
> See http://collector.zope.org/CMF for bug reports and feature requests
> 
> _______________________________________________
> Zope-CMF maillist  -  Zope-CMF@zope.org
> http://mail.zope.org/mailman/listinfo/zope-cmf
> 
> See http://collector.zope.org/CMF for bug reports and feature requests

-- 

Paul Winkler
http://www.slinkp.com
Look! Up in the sky! It's BRAINDEAD GIRLFRIEND!
(random hero from isometric.spaceninja.com)