[ZODB-Dev] Recovering from BTree corruption

[Alan Runyan]

> And, as you said in another node, the BTree folder actually loves in
> the resources database.

Correct the BTree is in /plone/resources/files to be exact.


> Cross database references are inherently weak.  A reference from a
> foreign database doesn't prevent an object from being treated as
> garbage.  So, if the only reference to an object is from a foreign
> database, then the object is considered garbage.  It doesn't sound
> like this is what's affecting you.  The cross-database reference is
> to the BTree.  It sounds like the internal references are within
> database.

Well.  Someone could have 'copy/pasted' a file from the content database
into the resources/files database.  That could have been one issue.

> >   - checkbtrees.py
> >   - fstest.py
>
> There's an fsrefs script that checks internal references I believe.

fsrefs.py shows loads of problems in both the data.fs and the resources.fs.
probably > 200 entries per database. i.e.

oid 0xD87110L BTrees._OOBTree.OOBucket
last updated: 2007-09-04 14:43:37.687332, tid=0x37020D3A0CC9DCCL
refers to invalid objects:
        oid ('\x00\x00\x00\x00\x00\xb0+f', None) missing: '<unknown>'
        oid ('\x00\x00\x00\x00\x00\xb0N\xbc', None) missing: '<unknown>'
        oid ('\x00\x00\x00\x00\x00\xb0N\xbd', None) missing: '<unknown>'
        oid ('\x00\x00\x00\x00\x00\xd7\xb1\xa0', None) missing: '<unknown>'
        oid ('\x00\x00\x00\x00\x00\xc5\xe8:', None) missing: '<unknown>'
        oid ('\x00\x00\x00\x00\x00\xc3\xc6l', None) missing: '<unknown>'
        oid ('\x00\x00\x00\x00\x00\xc3\xc6m', None) missing: '<unknown>'
        oid ('\x00\x00\x00\x00\x00\xcahC', None) missing: '<unknown>'
        oid ('\x00\x00\x00\x00\x00\xaf\x07\xc1', None) missing: '<unknown>'

My questions are:

 - I imagine if there are 'invalid' references this is considered "corruption"
   or "inconsistency"?

  - How do I tell if something is a reference to another database?

  - Having these invalid references, is this common to  ZODB applications?

> Possibly, there's a backup that has data records for the missing OIDs.

Going to ask hosting company to pull up backups for the past few weeks.
But how i'm going to find this other than "seeing if the folder allows me
to iterate over the items" is not throwing POSKeyError.  Does that sound
like a decent litmus test?

-- 
Alan Runyan
Enfold Systems, Inc.
http://www.enfoldsystems.com/
phone: +1.713.942.2377x111
fax: +1.832.201.8856