[ZODB-Dev] Recovering from BTree corruption

[Dieter Maurer]

Alan Runyan wrote at 2007-9-10 09:34 -0500:
> ...
>While debugging this I had a conversation with sidnei about mounted
>databases.  He recalled that if your using a mounted database you
>should not pack.  If for some reason your mounted database had a cross
>reference to another database and somehow you had a dangling reference
>to the other database it would cause POSKeyError.

BTrees are actually directed acyclic graphs (DAGs) with two node types
"tree" (internal node) and "bucket" (leaf).

Beside its children, a "tree" contains a link to its leftmost
bucket. Beside its keys/values, a "bucket" contains a link to
the next "bucket".

When you iterate over "keys" or "values", the leftmost bucket
is accessed via the root's leftmost bucket link and then
all buckets are visited via the "next bucket" links.
Your description seems to indicate that you have lost a
"next bucket" link.

If you are lucky, then the tree access structure (the children links
of the "tree" nodes) is still intact -- or if not, is at least
partially intact. Then, you will be able to recover large parts
of your tree.


You have two options:

  * reconstruct the tree from its pickles.

    This is the way, the checking of BTrees works.

  * Determine the last key ("LK") before you get the "POSKeyError";
    then use the tree structure to access the next available
    key. You may need to try ever larger values above "LK"
    to skip a potentially damanged part of the tree.


I would start with the second approach and switch to the first one
when it becomes too tedious.



-- 
Dieter