Hello, I've run into a misterious issue while evolving generations from an old DB. Quick fact is that it seems like a BTree kept an object reference to an object which was deleted from it. I'll try to explain/dig into below. BTrees source is of revision 75706, running on win32, compiled with mingw32. The evolution is about totally removing a utility because it's source will be ripped out in the next generation. It's in z3c.vocabulary. The utility contains objects in BtreeContainers recursively. Problem was that the loop at #kill ALL should have removed all object references from IntIds. For the first look it removed all, the loop at #clean intids never found one (toremove was always empty). All seemed fine, until the next generation came along and tried to touch all objects in ZODB. (ZODB was packed first) And *failed* on an instance of the utility. (source was gone) Debugging that found utility with http://www.zopelabs.com/cookbook/1114086617 clearly showed that it is still referenced by the IntIds.ids OIBTree, though impossible to reach via IntIds.ids.keys() or IntIds.ids.values(). What helped was to recreate both IntId BTrees from scratch, like at #recreate intid trees. That and packing made the utility finally go away. Anybody noticed something like this? -------------------------------------- def evolve(context): """Evolve the ZODB. - remove IVocabularyManager utilities """ root = getRootFolder(context) storage = context.connection._storage for site in findObjectsProviding(root, interfaces.IRecruiterSite): originalSite = hooks.getSite() hooks.setSite(site) sm = site.getSiteManager() if sm['default'].has_key('vocabularyManager'): vm = sm['default']['vocabularyManager'] ##kill ALL #remove all items from the utility to remove them from IntIds for k,v in list(vm.items()): for kk, vv in list(v.items()): for kkk, vvv in list(vv.items()): del vv[kkk] del v[kk] del vm[k] sm.unregisterUtility(vm, IVocabularyManager, '') del sm['default']['vocabularyManager'] if sm['default'].has_key('ids'): #clean intids iidutil = sm['default']['ids'] toremove = [] for id in iidutil: obj = iidutil.getObject(id) if not belowsite(obj, site): toremove.append(obj) if obj.__module__.startswith("z3c.vocabulary"): toremove.append(obj) for k, id in iidutil.ids.items(): obj = iidutil.getObject(id) if obj.__module__.startswith("z3c.vocabulary"): toremove.append(obj) if k.object.__module__.startswith("z3c.vocabulary"): toremove.append(k.object) for obj in toremove: iidutil.unregister(obj) #recreate intid trees: #because it seems like sometimes some buckets or whatever #reference objects once they are long gone ids = iidutil.ids refs = iidutil.refs iidutil.ids = iidutil.family.OI.BTree() iidutil.refs = iidutil.family.IO.BTree() for k,v in ids.items(): iidutil.ids[k] = v for k,v in refs.items(): iidutil.refs[k] = v ----------------------- #next generation touching all objects def evolve(context): """Evolve the ZODB. """ storage = context.connection._storage next_oid = None while True: oid, tid, data, next_oid = storage.record_iternext(next_oid) modname, classname = get_pickle_metadata(data) obj = context.connection.get(oid) # Make sure that we tell all objects that they have been changed. Who # cares whether it is true! :-) obj._p_activate() obj._p_changed = True if next_oid is None: break -- Best regards, Adam GROSZER mailto:agroszer@gmail.com -- Quote of the day: When a deep injury is done us, we never recover until we forgive. - Alan Paton