[ZODB-Dev] Obtaining a subset of BTree items

[Shane Hathaway]

On Wed, 15 Oct 2003, Christian Robottom Reis wrote:

> Let's see if anybody has run into this before. I'm trying to "filter" an
> IOBTree by a set of keys. I can cook the keys into an IOSet, but how can
> I retrieve the values that correspond to that set of keys?
> 
> At the moment, I'm doing
> 
>     [tree[i] for i in set]

Here's a trick I used.  For some uses, it's much faster than a list
comprehension (or any sort of Python iteration.)

>>> from BTrees.IOBTree import IOBTree, IOSet, difference
>>> set = IOSet([1,2])
>>> tree = IOBTree({1:'a', 2:'b', 3:'c', 4:'d'})
>>> complement = difference(tree, set)
>>> d = difference(tree, complement)
>>> list(d.items())
[(1, 'a'), (2, 'b')]

The trick is that difference() returns a bucket rather than a set.  
union() and intersection(), by contrast, return sets.  Although the
computation is more complex, it's all done in C.  In my tests, this method
came out faster for large sets (50,000 items or so) than even the most
optimized Python iteration.

I figure this is sufficient until BTrees grow some sort of map operation.

Shane