[Zope] Poor man's (application level) database replication?

Steve Spicklemire steve@spvi.com
Thu, 30 May 2002 16:17:24 -0500


Hi Folks,

This might be the world's dumbest idea... but I'm close enough to 
thinking that it's "OK" to start wasting time on it, and I want to get 
some feedback before I actually waste very much time. ;-)  It goes like 
this: I need to be sure that my SQL data is replicated somewhere. 
Replication seems to have spotty support among the free database, and 
it's also fairly complicated to set up. Mere mortals would like 
something easy, and dumb, to help out in cases where these two obstacles 
are a significant hurdle. ZSQLMethods required you to choose a database 
adaptor, and database adaptors generally permit you to connect only to a 
single database. I'm thinking that I could cook up a simple "db-tee" 
adaptor that keeps track of two things:

1) a list of "real" database adaptors that it knows about and their 
current "status" (OK/broken)
2) a callable "policy" object (python script, external method, ZClass?) 
that can work with it's list of DAs to implement the replication by 
directing the iteration process.

The basic plan is that a ZSQLMethod attaches itself to the tee, and 
calls the tee for database access. The tee then uses it's policy object 
to iterate through the list of database adaptors. For read operations 
(e.g., SELECT) the policy would use a "continue until successful" 
strategy, while write operations would use a "iterate over all DAs" 
strategy. Even the choice of which adaptors to iterate over could be 
variable (e.g., depend on the value of ZEO_CLIENT or somesuch). This way 
reading from the database(s) would be faster (since each ZEO_CLIENT 
could use a different DA as the first DA to check) but writes would be 
slower (since each DA would need to be called in turn to get 
replication). But, in my experience, writing is much less frequent than 
writing, so maybe that's OK? Also, the policy object could decide how to 
handle failures (on read: skip and set status to "broken", on write: 
raise an exception, or notify someone, or whatever you like).

Anyway.. I'm just wondering if anyone has toyed with this sort of plan, 
or has come up with something better?

thanks!
-steve