[ZODB-Dev] ConflictError and POSKeyError with 2.6 CVS :-(

[Andrew Sydelko]

On Wed, 16 Oct 2002 11:26:01 -0400 jeremy@zope.com (Jeremy Hylton) wrote:

> >>>>> "AS" == Andrew Sydelko <andrew@sydelko.org> writes:
> 
>   AS> On Wed, 16 Oct 2002 09:58:13 -0400 jeremy@zope.com (Jeremy
>   AS> Hylton) wrote:
>   >> Chris,
>   >>
>   >> I just checked in a little script that should report on dangling
>   >> references in a storage.  It scans the entire database and prints
>   >> a little report like this for a corrupted storage:
> 
>   AS> I decided to run your fsrefs.py on our Data.fs and it came back
>   AS> with:
> 
>   AS> # /data/www/Zope.10/bin/python fsrefs.py Data.fs
>   AS> Traceback (most recent call last):
>   AS>   File "fsrefs.py", line 79, in ?
>   AS>     main(sys.argv[1])
>   AS>   File "fsrefs.py", line 68, in main
>   AS>     data, serial = fs.load(oid, "")
>   AS>   File
>   AS>   "/data/www/Zope.10/src/Zope-2.5.1-src/lib/python/ZODB/FileStorage.py",
>   AS>   line 582, in load
>   AS>     return self._load(oid, version, self._index, self._file)
>   AS>   File
>   AS>   "/data/www/Zope.10/src/Zope-2.5.1-src/lib/python/ZODB/FileStorage.py",
>   AS>   line 569, in _load
>   AS>     return _loadBack(file, oid, pnv)
>   AS>   File
>   AS>   "/data/www/Zope.10/src/Zope-2.5.1-src/lib/python/ZODB/FileStorage.py",
>   AS>   line 2137, in _loadBack
>   AS>     doid, serial, prev, tloc, vlen, plen =
>   AS>     unpack(">8s8s8s8sH8s", h)
>   AS> struct.error: unpack str size does not match format
> 
>   AS> Is that a bad thing?
> 
> It's not a good thing.  It means there's an object in your database
> that can't be loaded.  I didn't consider that possibility when I wrote
> this script, although it makes sense to report that as one class of
> error.
> 
> Would it be possible to get a copy of your Data.fs and use it to
> improve the script?

Sure, it's only 7.7GB...

> I can't really say how bad a problem this is.  It may be some object
> that you're never going to want to load anyway.  You might run
> fstest.py over the file to make sure there isn't low-level corruption
> of the file.

# /data/www/Zope.10/bin/python /data/www/Zope.10/Zope/utilities/ZODBTools/fstest.py /data/www/Zope.10/Zope/var/Data.fs
1607487 object serialno 0x033b65b153b3006e does not matchtransaction id 0x033b6ad19ad6673b

Oh, and the database wouldn't do a 30-day pack, only a 0-day pack.

--andy.