[Zope] fsrecover.py won't pack

[Andy Altepeter]

Hi Tim,

Thank you for your response.  fsrefs.py raised the same error I reported 
earlier about unpickling.  I ended up:
1) Creating a new instance_home using zope-2.5.0-final
2) using zopectl debug connected to the bad database, I exported all my 
objects.  For folders that couldn't be exported, I started 'drilling down' 
until I found the object that was causing problems.  There were about 5 such 
objects in my database.
 - Some objects (folderish) could be exported, but then would raise a POSKey 
error upon import, so I drilled down in those folders as well.
3) I imported each .zexp into the new database.

This process was time consuming, but it appears to have recreated the ZODB in 
a sane state.  I think it's time to look into using repozo for backups ;-)

Andy

On Monday 08 August 2005 05:45 pm, Tim Peters wrote:
> [Andy Altepeter]
>
> > I ran fsrecover.py on a copy of my Data.fs.  It found no errors, but then
> > when it attempted to pack the file, it raised an unpickleable error, with
> > this traceback:
>
> ...
>
> >  File "/export1/Zope-2.7.5-final/lib/python/ZODB/referencesf.py", line
> > 38, in referencesf
> >    raise ValueError, 'Error unpickling, %s' % p
> > ValueError: Error unpickling, and the rest is presumably the pickle.
> >
> > I receive this same error when trying to pack using the ZMI.  Any idea
> > what's going on?
>
> It suggests data corruption in your Data.fs file (assuming you're
> using FileStorage).  Please read this for background info:
>
>     http://zope.org/Wikis/ZODB/FileStorageBackup
>
> Because fsrecover.py completed without error, best guess is that
> fstest.py (described in the link above) would also complete without
> error.  However, neither of those can detect "higher-level" corruption
> inside serialized object state (see the link for more words).  Running
> fsrefs.py (see the link again) can identify most such corruption.  So
> run it and see what it says.  There is no wholly safe and automatic
> way to recover from frsrefs.py-level corruption (short of reverting to
> a good backup), so don't expect this to go easily.
>
> Note that better versions of these tools exist in more recent versions
> of ZODB (IIRC, you said you were running Zope-2.7.2, which corresponds
> to ZODB 3.2.3 -- ZODB 3.2.9 is the current production 3.2 release).