[Zope] Will ZEO solve my problem?

[Kyler B. Laird]

On Mon, 02 Jul 2001 13:58:23 +0100 you wrote:

>Do you have a plan for replicating data onto the second ZEO server?

Yeah, but it's not pretty.

>We
>are looking to implement a similar system and there seems to be no
>obviously good choice.

That's what I determined about a week ago.
I was sad to see that work on Replicated
Storage had been halted, but I'm not sure
that it was the ideal solution for my
needs.

Then I had a little problem and became more
intimate with the Data.fs file.

Again, this is *not* pretty...

Right now, our Zope server moves around (It
even uses DHCP sometimes.) and our HTTPS
server stays in one place.  They're even in
different buildings.  I use the HTTPS
server to convert from HTTPS to HTTP and to
handle some other things like automatic
selection of the source port (for WebDAV
things).

When my Zope host comes up, it sets up an
SSH session to the HTTPS server.  As part
of the session, it makes tunnels for the
ZopE ports.  Now I also have it run a
command.  It's essentially this.
	tail -f -n +0 Data.fs | ssh proxyhost "cat >somepath/Data.fs"
(The real server name, account and tunnel
info are all in the SSH config.  Instead of
"Data.fs", for now I use a filename that
has the current date in it.  That gives me
multiple versions.)

This gives me an up-to-the-second copy of
the database.  If the building with the
Zope server is suddenly destroyed, I should
not lose anyuthing.

My plan is to do something similar between
two ZEO servers.  One will run and stream
its updates to the other.  If the primary
server fails, I will switch to the other
one (by notifying the HTTPS proxy or by
twiddling the SSH sessions).

Without any modifications, the secondary
server would have to start Zope and read
through the Data.fs before becoming active.
That's quite a delay for us.  My plan is to
modify the Zope startup so that it does
something similar to a "tail -f" on Data.fs
so that it can build its internal database
as it goes along (mirroring the primary
server).  I would then send a signal to it
to halt reading (when it reaches EOF) and 
start serving.

I might be able to get away with *not*
modifying the Zope startup by using a FIFO.
I would either write directly to it from my
SSH process or use tail to duplicate the
streamed file.  When the primary server
dies, I'll kill whatever is writing to the
FIFO (thus closing the write end) and Zope
should discontinue reading and start
serving.

Of course, this is going to require some
manual intervention before the primary
server is made available again.  For now,
I am happy to have it killed and forgotten
until someone puts everything right.  If
my primary and secondary ZEO servers are
identical (in terms of performance), I'll
probably just flip-flop them, and call the
running server the primary.

After I fix the dead server, I'll replace 
its Data.fs file with a stream of the
running server's file.  It will then be
ready to go as the fail-over server.

Note that although I am streaming the
Data.fs file now, everything else I'm
planning is untested.  I'm hoping to get
to it this week, but I've got some other
things in the way.

--kyler