[ZODB-Dev] BTrees and PersistentDict keys: bug or feature

[Casey Duncan]

On Thursday 14 August 2003 10:39 pm, Christian Reis wrote:
> On Thu, Aug 14, 2003 at 10:00:03PM -0400, Tim Peters wrote:
> > The docs I pointed you at before strongly recommend using simple, 
immutable
> > values, with well-defined __cmp__ methods, as BTree keys.  When you go
> > beyond that, you're trying things nobody understands.  For example, I have
> > no idea how a PersistentMapping will or won't compare itself against a
> > Python dict, and I wager nobody else does either.
> 
> To explain a tiny bit further, I'm using an OOBTree that maps
> dictionaries to objects. This is part of the indexing mechanism in
> IndexedCatalog, which is a simple indexing and query mechanism for the
> ZODB. This allows us to do queries like:
> 
>     "5 in mydict"

I'm not sure how using dictionaries as keys helps here (it just sounds evil to 
me ;^), but this is one place where the Python hashing rules for dictionary 
keys begin to make lots of sense. In Python you are forbidden from using 
simple mutable objects as keys since generating a stable hash for them is not 
possible unless you subclass and supply your own hash.

BTrees make it much easier to shoot yourself in the foot since keys must only 
be comparible, which mutable objects usually are. So the thing to note (no 
doubt you have realized this) is that when you use mutable objects as BTree 
keys (or set members) you are playing with fire and your are very likely to 
get burned.

> and so on as fast as BTree lookups can happen. It's a minor feature, but
> it's used in a couple of places in a couple of projects, and ever since
> I decided to clean up the handling of mutable attributes, I've been at
> war against corrupted BTrees. I've now resorted to writing simple
> wrappers around PersistentList and PersistentMapping that emit update
> signals every time they are mutated (which trigger unindexing and
> indexing of the values). I ran into this bug while I was debugging an
> odd problem with keys of mixed types ending up in the BTree, and I
> thought I would share it with the mailing list.
> 
> I've fixed this to make sure I only store dictionaries as keys. My
> wrapper class makes sure that the dicts are never mutated while being
> used as being keys, and so far check() has been reassuring.

Sounds like mongo pain for a "minor feature".

hoping-the-bandages-and-grafts-heal-soonly y'rs,

-Casey