[Zope] ZODB replication on a per-object basis?

[Petru Paler]

On Thu, May 11, 2000 at 09:44:27AM -0500, Brad Clements wrote:
> > > Based on this, it appears that ZSS does not replicate at this time.
> > 
> >     You're right, but notice the "at this time". Once it will be made
> > open source I'm pretty sure that someone will complete it.
> 
> Who is someone? Shouldn't that be us?

   I didn't mentioned who :)

> >    ZEO _does_ invalidation. Otherwise it would be useless.
> 
> I saw no mention of how CS invalidates Zope's internal cache based on 
> asynchronous information from ZSS. 

   I am pretty sure this is done. If you are not convinced, look at
the last week's CVS commits and see how special ZEO hooks are merged
into Zope.

> >    You want to do this too ? :) ZEO in its current form perfectly fits the
> > bill, because you _need_ a centralized site which the mirrors replicate.
> 
> I disagree. I think the performance would be bad. 
> 
> CS/ZSS is just an RPC implementation of FileStorage (it seems). 
> Therefore when Zope needs an object that is not in the in-memory 
> cache, CS has to execute an RPC call across the Internet to the 
> master copy -- slow..
> 
> Since the in memory cache has a limited size (I assume) and/or times 
> out old objects (I assume), statistically you're going to get hit with this 
> delay on the larger objects (Python packages) whenever you download. 

   I agree with you, but the local cache is very big (disk based I
think -- anyone from DC care to comment ?). ZEO is aggresively
optimized for the many-read, less-write case.
   The only expensive operations are writes, which must cause
invalidation of all the caches (though I hope it only invalidates the
changed OID and not the whole cache; I haven't seen the ZEO source yet
so I can't assume more).

> If you use a distributed file system instead, you don't have this problem. 
> Each site can either be a replicating server.. or in the case of Coda, 
> each remote site can enable "file hoarding" in their client to keep local 
> copies of all objects (files) ..
> 
> You definately want local copies of all the data. But these copies need 
> to be automatically updated from "the master" (in this application there 
> is one master, but in Coda that does *not* have to be the case).
> 
> We could do a poor-man's job at this now by running both an FTP server 
> and a Zope server pointing to those FTPable files. The FTP server can 
> mount a distributed file system (afs, coda)... Then nightly just export the 
> Zope database and replicate that to all the clients.
> 
> But this isn't quite what I want...
> 
> 
> 
> Brad Clements,                bkc@murkworks.com   (315)268-1000
> http://www.murkworks.com                          (315)268-9812 Fax
> netmeeting: ils://ils.murkworks.com               AOL-IM: BKClements

-Petru