[Zope] BTree listing by type

[Casey Duncan]

On Tue, 2 Dec 2003 16:59:29 +0100
"Andre Meyer" <a.meyer at hccnet.nl> wrote:

> Dear Zopistas
> 
> Here a question wrt. BTrees:
> 
> When I create a IOBTree object (for example) and add objects of different kinds (classes) to it, how can I retrieve a list of all objects of a certain type? What I want is something similar to objectValues(type) in Folder. Does that exist for BTrees or do I need to use ZCatalog? How would that work?

BTrees are low-level data types (like Python dictionaries). ZCatalog is a mini database application (it uses many BTrees internally). So comparing them to one another is not terribly helpful.

The high-level solution Dieter suggested might help, but BTreeFolder2 searches by Zope meta_type, not class (Although meta_types are often just friendly names for classes). If that's not what you want, then you will need to keep your own separate index of class=>objects. This can be done using BTrees, but I will caution that you cannot reliably use a class as a BTree key because BTrees keys are internally ordered based on comparison and Python classes will not compare the same way across program runs.  Class comparison is based on identity which changes each time Python is run.

The best option I have found is to convert classes to strings which always compare in a predictable order. A simple way to do this is using the Python pickle machinery. Classes usually have short pickle strings, and you can easily get the class back out by unpickling the key. Another option would be for use the dotted path for the class as the key. This would take a bit more work to implement, but would make the keys more human-readable.

The next problem is storing the sets of objects per key. Since you already have an IOBTree I assume that means there is a unique integer identifier for each object. If that is the case then you can use an IITreeSet set as the value for each class key. IITreeSets are sets that are designed to scale to arbitrarily large sets of objects by breaking the set up internally into separate persistent buckets so that they can be individually loaded and unloaded from memory.

If you do decide to persue BTrees further, I would suggest the following recreational reading:

http://zope.org/Wikis/ZODB/FrontPage/guide/node6.html

Enjoy.

-Casey