[Zope] Re: data.fs corruption

[Michel Pelletier]

alan runyan wrote:
> 
> I hope you dont mind, but im cc'ing the list-- ppl need to see this
> 
> one of the scariest things i can imagine is a customer calling up saying the
> entire *monolithic* database is corrupt.  the entire workday would grind to
> a halt and someone would have to  jump through lots of hoops to just get the
> database (zope filesystem) back in order :((((  this is unacceptable
> (period) for some of the uses i had in store for Zope.

The risk here is identical to a corrupted ext2 filesystem, which is
monolithic.  This is also not much different that what other relational
databases provide, ie, your data in a prorietary format in one file. 
Some even use their own partition!

This is not to say that your comment is not valid.  But many of your
worries are mitigated every day for many 'monolithic' systems buy
ensuring clean system shutdowns, redundant information storage (like
RAID), the use of journaling filesystems, and the *regular backing up of
data*.

Further Data.fs files don't just up and corrupt themselves, just like
ext2 filesystems do not randomly corrupt themselves.  ZODB has evolved
over years and is quite stable.  Something has to be done to have put
the Data.fs file in a corrupt state, such as the partition running out
of space (Note Zope is smart enough to detect incomplete transaction due
to insufficient space on the next startup) or unclean system shutdowns
(where the file buffers do not get flushed).  Once again these are risks
that are mitigated by various means such as those above (and perhaps
some duct-tape on the power switch).
 
> is zope going to support multiple databases? 

Yes.

> jim f. said the ZODB is a
> SPOF -- are there any plans on fixing this? 

No, it's not broken.  It is no more so a single point of failure than
any other non-redundant component of your system.

> is there anyway to 'block all
> writes' while backing up the ZODB?

Yes, block all requests, or at least those requests that you know will
write data.

> w/o taking the entire website
> down/mostly reads anyways?

No, just like backing up any system, there is a give an take here.  If
you want to back up a UNIX box but still work on it, then you run the
risk of getting a backup that is not entirely current.  If you want to
guarentee currency, then boot down to single user mode.  Same situation
in Zope, if you want absolute currency, then you must do the equivelent
of bring Zope to 'single user mode', ie, don't let anyone log in.  Note
that a Zope backup is still VALID if users are still writing data to it,
it just might not be absolutly CURRENT.

> is there a way you can 'scan' to see if the ZODB is intact? i.e. having a
> process scan the ZODB and return whether or not it is valid?  what was the
> 'offending' transaction?

It is true that Data.fs corruption is difficult to diagnose and fix, and
that ext2 has a fancy tool (fsck) to help you with your linux filesystem
and Zope should also provide such a tool.  Ty Sarna's 'Tranalyzer' is a
good place to start.

-Michel


> 
> thanks
> ~alan
> 
> ----- Original Message -----
> From: "M. Adam Kendall" <mak@kha0s.org>
> To: "alan runyan" <runyaga@thisbox.com>
> Sent: Sunday, February 20, 2000 1:52 AM
> Subject: RE: data.fs corruption
> 
> > Of course, you probably aren't going to like this answer,
> > but I spent about 4.5 hours with the Tranalyzer product,
> > and a good hex editor hacking away at my Data.fs.  The good
> > news: I recovered all my data.  The bad news: I have alot
> > less hair now..
> >
> > I basically used Tranalyzer to figure out where it blew up,
> > and then from the comments in FileStorage.py in the Zope source
> > I was able to figure out the file format for the Data.fs.  At
> > least I knew how many bytes record/field lengths were supposed
> > to be.  Then using the hex editor I manipulated the Data.fs until
> > I had ripped out the offending transaction record.
> >
> > Worked like a charm but was a very long drawn out process..
> > I would say that Digital Creations needs to provide a tool
> > for easy data recover from corrupt Data.fs.
> >
> > Adam
> >
> 
> _______________________________________________
> Zope maillist  -  Zope@zope.org
> http://lists.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists -
>  http://lists.zope.org/mailman/listinfo/zope-announce
>  http://lists.zope.org/mailman/listinfo/zope-dev )