[ZODB-Dev] corrupt filestorages - eep :-(

[Chris Withers]

Hi All,

I'm dealing with what I suspect are a couple of corrupt filestorages.
One is packed every time content is loaded by a batch process and the 
other is never packed. The UnPacked storage is mounted as the root and 
then the Packed one is mounted in to /content.

Running fstest.py on both .fs files elicits no output, so I assume that 
means it thinks things are okay?

Running fsrefs.py against Packed.fs gives me a nice variety of errors 
[1] but I'm not sure what these mean. Is fsrefs.py aware of mutliple 
mounted storages? If so, then what do these errors mean?

Running fsrefs.py against UnPacked.fs gives me a similar variety of 
errors [2] but eventually bombs out with:

# python2.4 /usr/lib/zope2.9/bin/fsrefs.py var/UnPacked.fs > 
unpacked_errors.txt
Traceback (most recent call last):
   File "/usr/lib/zope2.9/bin/fsrefs.py", line 157, in ?
     main(path)
   File "/usr/lib/zope2.9/bin/fsrefs.py", line 130, in main
     refs = get_refs(data)
   File "/usr/lib/zope2.9/lib/python/ZODB/serialize.py", line 687, in 
get_refs
     data = oid_klass_loaders[reference_type](*args)
KeyError: 'n'

I'm guessing this isn't good ;-) What does that KeyError mean?

What would be the suggested way of proceeding?

cheers,

Chris

[1] a small selection, the file they came from is 2087 lines long:

oid 0x0L Persistence.mapping.PersistentMapping
last updated: 2006-07-26 22:06:53.910524, tid=0x367150EE604AB11L
refers to invalid object:
         oid ('\x00\x00\x00\x00\x00\x00\x00\x01', None) missing: '<unknown>'

oid 0xAC5L OFS.Folder.Folder
last updated: 2008-01-19 12:02:33.000289, tid=0x3732E128CCD1D99L
refers to invalid objects:
         oid ('\x00\x00\x00\x00\x00\x00$\xaf', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x00Ba', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x00Bb', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x00Bc', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x003\xef', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x00Bd', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x00I\x06', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x00Be', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x00Bf', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x00$\xb0', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x00I\x05', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x00"Z', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x003\xee', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x003\xed', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x003\xeb', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x00$\xb1', None) missing: '<unknown>'

oid 0x33EBL Products.PageTemplates.ZopePageTemplate.ZopePageTemplate
last updated: 2006-07-26 22:46:07.670653, tid=0x367153620BA6611L
refers to invalid object:
         oid ('\x00\x00\x00\x00\x00\x003\xec', None) missing: '<unknown>'


[2] a small selection, the file they came from is 1298 lines long:

oid 0x0L Persistence.mapping.PersistentMapping
last updated: 2006-07-26 22:04:21.420770, tid=0x367150C5B653166L
refers to invalid objects:
         oid ('\x00\x00\x00\x00\x00\x00\x00\x01', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x01$\xfc', None) missing: '<unknown>'

oid 0x1L OFS.Application.Application
last updated: 2008-02-12 10:53:11.840171, tid=0x373B4CD32849FCCL
refers to invalid objects:
         oid ('\x00\x00\x00\x00\x00\x00\xd8\x11', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x00\xb7\xf8', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x00\x86\x10', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x02\xb4\x0b', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x01\xdaQ', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x00\x85\xff', None) missing: '<unknown>'
         oid ('\x00\x00\x00\x00\x00\x00\xd8\x07', None) missing: '<unknown>'


-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk