[ZODB-Dev] BTrees and PersistentDict keys: bug or feature

[Christian Reis]

On Fri, Aug 15, 2003 at 09:25:39AM -0400, Casey Duncan wrote:
> > I'm suffering from lack of oxygen (hopefully). The important query here
> > is, or course:
> > 
> >     "mydict == {foo: bar}"
> > 
> > -- which is handled correctly (AFAICT, and I can't tell much yet because
> > I still need to test a lot) by simple comparison.
> 
> Here are some thoughts, both involving reducing the complexity by converting 
> the mutable objects to an immutable representation:
> 
> A list and a dict can both be represented in immutable form. The former as a 
> tuple and the later as a tuple of key/value pair tuples.

My latest change* to IC uses a tuple as the key for SequenceIndexes
(which covers tuples and lists). For MappingIndex I store a *copy* of
the dictionary as the key. 

I think I'm fairly safe from mutation in both cases. The mapping itself
requires using a special class (the IndexedDict I mentioned before) that
notifies the index that the content has changed. This is a bit slow in
theory, but in testing our applications it seems acceptable (and I don't
see another solution for transparent index updates -- do you?)

> Have you considered converting the values to an immutable form and using that 
> as a key? This might have another advantage (or maybe disadvantage), which is 
> that all list-like things and all dict-like things would get a LCD 
> representation which would compare consistently.

Yep. That's what my change to IC did (where I uncovered these odd
problems with comparisons between PM and {}).

[*] Johan actually wrote more than half of it.

> So PersistentMapping({1:2, 2:3}) would become ((1,2),(2,3)) as would {1:2, 
> 2:3}. Determining what is a mapping could be done with some introspection 
> logic (like looking for an "items" method) or by some explicit configuration. 
> Now, this would still fall down if the key/values themselves were mutable.

> So, in that case I might suggest instead pickling the object and using that as 
> a key. Then all the keys would be strings. This would never consider 
> PersistentMappings equal to dicts though, but maybe that's what you want. I 
> used this technique recently when I needed to use arbitrary types/classes as 
> BTree keys.

The hard part here isn't converting stuff into a stable key format, but
really making sure the index stays up-to-date with the data. How did you
tackle that part of the problem?

The pickle trick is interesting, I'll hold on to this for a bit so I can
test the idea.

Take care,
--
Christian Reis, Senior Engineer, Async Open Source, Brazil.
http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL