Attention: cross post We have run in an apparent "BTrees" bug (with the "BTrees" version included in Zope 2.5.1). We try to insert a value into an "BTrees.OOBTree.OOBTree" instance with tree[k]= v and get an "exceptions.KeyError on 13384379A0pfvqTmtEQ". The code comes from Zope's "Transience" implementation. The reported key is some session key. We are not yet sure, however, whether it is the key we are just trying to insert. Reading the "News.txt" of ZODB-3.2a1, this bug may not yet be known. We will install the "BTrees" package from ZODB-3.2a1 and see whether the problem disappears. I will report back... Dieter
Tim has fixed lots of bugs in the BTrees code since 2.5.1. Also, there were cases in the Transience implementation where I was iterating over the .keys()/.items()/.values() of an OOBTree or IOBTree and also mutating its contents, which is disastrous. Is this one of those cases? On Fri, 2003-01-24 at 03:13, Dieter Maurer wrote:
Attention: cross post
We have run in an apparent "BTrees" bug (with the "BTrees" version included in Zope 2.5.1).
We try to insert a value into an "BTrees.OOBTree.OOBTree" instance with
tree[k]= v
and get an "exceptions.KeyError on 13384379A0pfvqTmtEQ".
The code comes from Zope's "Transience" implementation. The reported key is some session key. We are not yet sure, however, whether it is the key we are just trying to insert.
Reading the "News.txt" of ZODB-3.2a1, this bug may not yet be known.
We will install the "BTrees" package from ZODB-3.2a1 and see whether the problem disappears. I will report back...
Dieter
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Dieter Maurer]
We have run in an apparent "BTrees" bug (with the "BTrees" version included in Zope 2.5.1). ... We will install the "BTrees" package from ZODB-3.2a1 and see whether the problem disappears. I will report back...
There are two new kinds of sanity checks you can make there: 1. somebtree._check() This is a new method on BTree objects, which does extensive checks of internal inter-pointer consistency. It raises AssertionError if internal pointers have gotten into an inconsistent state. This used to be possible in the presence of (unlikely mixtures of) inserts and deletes. 2. from BTrees.check import check check(somebtree) This is a brand new checker that does a different kind of consistency check, verifying subtle invariants involving the key *values* (not the internal C pointers) in a BTree (or TreeSet). Again, it raises AssertionError if anything is wrong there. If anything is wrong, from BTrees.check import display display(somebtree) displays the internal bucket structure of a BTree, to stdout. It can clarify the nature of the damage. There are no known ways of creating a damaged BTree anymore, short of (so far) hypothetical scenarios involving multiple processes somehow managing to get hold of stale data without realizing it. 3.2a1 fixes a newly-discovered but extremely unlikely bug in BTree conflict resolution. I don't know that it's ever been seen in real life (it was provoked by a multithreaded stress test I wrote) -- if it ever did happen, the symptom would have been a segfault, not silent BTree corruption.
We try to insert a value into an "BTrees.OOBTree.OOBTree" instance with
tree[k]= v
and get an "exceptions.KeyError on 13384379A0pfvqTmtEQ".
That's indeed peculiar.
Attention: cross post Dieter Maurer wrote at 2003-1-24 09:13 +0100:
We have run in an apparent "BTrees" bug (with the "BTrees" version included in Zope 2.5.1).
We try to insert a value into an "BTrees.OOBTree.OOBTree" instance with
tree[k]= v
and get an "exceptions.KeyError on 13384379A0pfvqTmtEQ".
The code comes from Zope's "Transience" implementation. The reported key is some session key. We are not yet sure, however, whether it is the key we are just trying to insert.
Reading the "News.txt" of ZODB-3.2a1, this bug may not yet be known.
We will install the "BTrees" package from ZODB-3.2a1 and see whether the problem disappears. I will report back...
It has been a false alarm. Sorry! The same problem occured with the new BTrees module. Therefore, I checked the error again and found, that the problem was not caused by "tree[k] = v" but by a nearby "del tree[k]" (which, of course, is entitled to raise "KeyError"). When we had analysed the error for the first time, we happened to look at some slightly outdated source. Viele Grüße Dieter
[Dieter.Maurer@Haufe.de]
It has been a false alarm. Sorry!
The same problem occured with the new BTrees module.
Therefore, I checked the error again and found, that the problem was not caused by "tree[k] = v" but by a nearby "del tree[k]" (which, of course, is entitled to raise "KeyError").
When we had analysed the error for the first time, we happened to look at some slightly outdated source.
I appreciate the follwup, Dieter! I've gone half blind searching the source for any way a BTree __setitem__ could have raised KeyError. I didn't find one, even if the BTree were corrupted. But that doesn't mean it couldn't happen <wink>, so knowing it wasn't actually happening is a relief.
participants (4)
-
Chris McDonough -
Dieter Maurer -
Dieter.Maurer＠Haufe.de -
Tim Peters