[ZODB-Dev] Recovering Corrupt Database

[Cody Smith]

Actually we did have (nightly) backups, but before the disk was failing, it
was merely behaving strangely.  When we started making tar.gz backups, tar
made some strange complaint (like "file shrunk by xyz bytes"), but didn't
fail to make the archive, and md5sums matched, so I was more than happy to
trust the backups.

Can you really make live backups of Data.fs?

Well, I'm trying the upgrade now, we'll see if it works.

Cody

----- Original Message -----
From: "Tino Wildenhain" <tino@wildenhain.de>
To: "Cody Smith" <smithcc@uclink.berkeley.edu>; <zodb-dev@zope.org>
Sent: Wednesday, October 30, 2002 4:15 Doh!
Subject: Re: [ZODB-Dev] Recovering Corrupt Database


> Hi Cody,
>
> I know, this does not help you at the moment,
> but you plan to backup now on a regular basis,
> dont you?
>
> Backup is easy for Data.fs - just copy it to somewhere else
> even when zope is running. cron and bzip2 is your friend :)
>
> Regards
> Tino
>
> --On Mittwoch, 30. Oktober 2002 02:53 -0800 Cody Smith
> <smithcc@uclink.berkeley.edu> wrote:
>
> > We're running Zope 2.3.2, and haven't really had any problems with the
> > back end (which is my business, instead of the marketing team's
> > business), so I haven't had to fret over it recently.  Quite sadly, our
> > zope server suffered from some disk problems recently, and we lost a
> > chunk of data in the _middle_ of our Data.fs, and now Zope refuses to
> > start.  We get this error message:
> >
> > zope# ./start
> > 2002-10-30T10:37:40 PANIC(300) z2 Startup exception
> > Traceback (innermost last):
> >   File /opt1/Zope-2.3.2-src/z2.py, line 566, in ?
> >   File <string>, line 1, in ?
> >   File /opt/Zope-2.3.2-src/lib/python/Zope/__init__.py, line 110, in ?
> >   File ../lib/python/ZODB/FileStorage.py, line 308, in __init__
> >     (Object: /opt1/Zope-2.3.2-src/var/Data.fs)
> >   File ../lib/python/ZODB/FileStorage.py, line 1674, in read_index
> >   File ../lib/python/ZODB/FileStorage.py, line 219, in panic
> > CorruptedTransactionError: /opt1/Zope-2.3.2-src/var/Data.fs data record
> > exceeds transaction record at 300605488L
> >
> > At this point, I find the fsrecovery script and run it.  Unfortunately,
> > it, like the start script, fails.
> >
> > zope# python lib/python/ZODB/fsrecover.py var/Data.fs
> > Traceback (innermost last):
> >   File "lib/python/ZODB/fsrecover.py", line 94, in ?
> >     FileStorage.recover(sys.argv[1])
> >   File "lib/python/ZODB/FileStorage.py", line 1542, in recover
> >     file.truncate(npos)
> > NameError: npos
> >
> > Now I'm forced to look at the script myself to see why this is
happening,
> > and I note that, in the script, the last few lines before the error look
> > like this:
> >
> >     if pos < sz:
> >         npos = shift_transactions_forward(
> >             index, vindex, tindex, file, pos, opos,
> >             )
> >
> >     file.truncate(npos)
> >
> > Now it makes sense: if pos >= sz, npos is undefined.  So I add in a
> > debugging line and find that pos = 406821544 and sz = 406821544.  After
> > that, I add an else clause to set npos=opos (or was it pos, I don't
> > exactly remember), that takes 100MB of the size of Data.fs and rolls my
> > database back to summer 2001 (angering the marketing team, who insist
> > that my job isn't done).  Now I'm stuck, but it seems to me based on the
> > first error message that I could probably fix the problem by correcting
> > either the broken transaction record or the broken data record (which is
> > actually broken, I'm not sure), or both.  Would this help?  Is there
> > anything I can do to recover the database?
> >
> > Thank you _so_ much, if I don't fix this, the marketing team will
> > literally have lost a years worth of work, and no one will be happy.
> >
> > Cody
> >
> >
> > _______________________________________________
> > For more information about ZODB, see the ZODB Wiki:
> > http://www.zope.org/Wikis/ZODB/
> >
> > ZODB-Dev mailing list  -  ZODB-Dev@zope.org
> > http://lists.zope.org/mailman/listinfo/zodb-dev
>
>
>
> _______________________________________________
> For more information about ZODB, see the ZODB Wiki:
> http://www.zope.org/Wikis/ZODB/
>
> ZODB-Dev mailing list  -  ZODB-Dev@zope.org
> http://lists.zope.org/mailman/listinfo/zodb-dev