[ZODB-Dev] HA Setup and zodb index file of FileStorage

[Patrick Gerken]

On 9/25/06, Chris Withers <chris at simplistix.co.uk> wrote:
> Patrick Gerken wrote:
> > system with ZEO for an ERP5 deployment. In my case I don't need to
> > care for data replication, all is stored on a SAN considered HA by the
> > customer already.
>
> You run your live Data.fs off a SAN?
> That usually makes for interesting performance problems in the best case!

Well, yes, that is/was the idea when suddenly HA requests popped up.
You might be right, but I would still give it a try, whenever we would
need to access many many objects we use an SQL index, and nobody will
access too many objects. Most of them will probably lay around and
never touched again. Though we didn't start with profiling yet, I feel
pretty
safe to say that the typical number of users for one zeo client will
not try to access 1000 different objects.

> > So my data.fs and index and all that stuff will already be available
> > on my backup server.
>
> How does it measure "down"?

More important is the question what we will do if we think it is down.
Through I do not think about details currently.

> > The thing which scared me to hell was the "rumour" that it can happen
> > that the index can get corrupted and then everything has to be indexed
> > again.
>
> without ZRS, don't even try to have a "live" backup. Best you can aim
> for is a half-hour swapover if you have a decent sized (10-20GB)
> database. With ZRS, it may be less, but I remain to be convinced.
>
> > Given that writing a file and renaming a file can be considered
> > atomic, and that no solar winds or similar things can screw up my
> > filesystem, how can I screw up my index file?
>
> The index checking is likely pessemistic, as you've noticed.
> Optimisations, I'm sure, would be welcome, but you'd need to prove they
> can't result in dangerous corruption...

Lets see what can be done.

Thanks for your remarks!

best regards,

            Patrick