[ZODB-Dev] Re: [Zope-dev] [BUG] maybe yet unknown BTREES bug

[Tim Peters]

[Dieter Maurer]
> A "tree[k] = v" should never raise a "KeyError" whatever you are
> doing.

I agree, but if a BTree is internally inconsistent, it's hard to say what
can happen:  the data structure has gone insane at the C level.

> I skimmed the "News.txt" accompanying ZODB 3.2a1 for fixed BTrees bugs.
> I could not find a hint that ours had been covered.

I've never before heard of your particular symptom, but we know it was
possible for BTrees to become internally inconsistent in the version of Zope
you're using.  If you can, please use the checking methods I explained last
time to see whether this BTree is in fact corrupted.

[Chris McDonough]
> You would think so, but if you do this:
>
> for k in btree.keys():
>    btree[k] = 1
>
> ... what comes back from "keys()" is not a list or tuple, its a
> BTreeItems object, which (from what I understand) itself is a data
> structure with pointers into "btree".

That's true.

> If you mutate "btree" while iterating over its keys in this way the
> pointers in the data structure get out of sync and don't do what you
> expect.

Also true.

> It is possible that a KeyError would be raised in this scenario (maybe
> not this exact scenario, but one a lot like it).

That one's a stretch <wink>.  Mutating while iterating may give surprising
results, but it shouldn't be possible to *damage* a BTree this way, and a
healthy BTree should never raise KeyError on "tree[k] = v", *unless* it's
the case that the type of the keys implement their own __cmp__ method that
may itself raise KeyError (in which case it's not the BTree methods that are
raising KeyError, but the user-defined __cmp__ method).  It may very well
raise KeyError on the *reference* "tree[k]", but Dieter is doing a
__setitem__ here, not a __getitem__:  if k "vanished" from the tree,
__setitem__ should merely create it again.

> Cases where I did this in the Transience package have been fixed in CVS.

Lots of bugs have indeed been fixed.

> Good luck in production!

Ditto!