Upgrade woes: BTreeFolder --> BTreeFolder2
I have a couple of outstanding issues following an otherwise successful upgrade from Zope 2.7.4 to 2.10.3-final. One key application used the old DocumentLibrary product (http://www.zope.org/Members/Kaivo/DocumentLibrary). I have since reimplemented most of the functionality and migrated out the content, but I am unable to delete the old instances of the 'DocumentStore' class. This class is based on the old BTreeFolder which, as far as I can tell, cannot be be used in current versions of Zope. Starting Zope with the old BTreeFolder fails: Traceback (most recent call last): File "/usr/local/zope/lib/python/OFS/Application.py", line 703, in import_product product=__import__(pname, global_dict, global_dict, silly) File "/usr/local/zope/client0/Products/BTreeFolder/__init__.py", line 89, in ? import BTreeFolder File "/usr/local/zope/client0/Products/BTreeFolder/BTreeFolder.py", line 91, in ? from BTree import BTree ImportError: No module named BTree Swapping in BTreeFolder2 allowed me to access existing instances of DocumentLibrary, however attempting to delete the old instances of DocumentStore fails with the following error: Traceback (innermost last): Module ZPublisher.Publish, line 119, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 42, in call_object Module OFS.ObjectManager, line 524, in manage_delObjects Module OFS.ObjectManager, line 379, in _delObject Module zope.event, line 23, in notify Module zope.component.event, line 26, in dispatch Module zope.component._api, line 130, in subscribers Module zope.component.registry, line 290, in subscribers Module zope.interface.adapter, line 535, in subscribers Module zope.component.event, line 33, in objectEventNotify Module zope.component._api, line 130, in subscribers Module zope.component.registry, line 290, in subscribers Module zope.interface.adapter, line 535, in subscribers Module OFS.subscribers, line 108, in dispatchObjectWillBeMovedEvent Module zope.app.container.contained, line 181, in dispatchToSublocations Module OFS.subscribers, line 88, in sublocations Module Products.BTreeFolder2.BTreeFolder2, line 368, in objectValues Module Products.BTreeFolder2.BTreeFolder2, line 347, in objectIds AttributeError: 'NoneType' object has no attribute 'keys' If I could install the original BTreeFolder in Zope 2.10.3 - if only briefly - would that enable me to delete the old instances? Other ideas? Thanks, Ken __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Ken Ara wrote at 2007-5-3 08:56 -0700:
I have a couple of outstanding issues following an otherwise successful upgrade from Zope 2.7.4 to 2.10.3-final. ... "/usr/local/zope/client0/Products/BTreeFolder/BTreeFolder.py", line 91, in ? from BTree import BTree ImportError: No module named BTree
Make your migration in the old Zope instance (where you have both the old "BTree" as the new "BTrees.XXBTree". Then copy the storage over to the new Zope instance.. -- Dieter
Thanks, Dieter. In retrospect, that looks like what I should have done, but unfortunately we no longer have access to the old machine. Everything else went so well - only now do we see that these 'dead' instances weigh in at about 1Gb. It does not appear critical today, but eventually we may need to do something about this... Could the solution be to redefine this product into something really simple and then kill it off? Here is the traceback again, that results from an attempt to delete a 'DocumentStore': Traceback (innermost last): * Module ZPublisher.Publish, line 119, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 42, in call_object * Module OFS.ObjectManager, line 524, in manage_delObjects * Module OFS.ObjectManager, line 379, in _delObject * Module zope.event, line 23, in notify * Module zope.component.event, line 26, in dispatch * Module zope.component._api, line 130, in subscribers * Module zope.component.registry, line 290, in subscribers * Module zope.interface.adapter, line 535, in subscribers * Module zope.component.event, line 33, in objectEventNotify * Module zope.component._api, line 130, in subscribers * Module zope.component.registry, line 290, in subscribers * Module zope.interface.adapter, line 535, in subscribers * Module OFS.subscribers, line 108, in dispatchObjectWillBeMovedEvent * Module zope.app.container.contained, line 181, in dispatchToSublocations * Module OFS.subscribers, line 88, in sublocations * Module Products.BTreeFolder2.BTreeFolder2, line 368, in objectValues * Module Products.BTreeFolder2.BTreeFolder2, line 347, in objectIds AttributeError: 'NoneType' object has no attribute 'keys' An error was encountered while publishing this resource. --- Dieter Maurer <dieter@handshake.de> wrote:
Ken Ara wrote at 2007-5-3 08:56 -0700:
I have a couple of outstanding issues following an otherwise successful upgrade from Zope 2.7.4 to 2.10.3-final. ...
"/usr/local/zope/client0/Products/BTreeFolder/BTreeFolder.py",
line 91, in ? from BTree import BTree ImportError: No module named BTree
Make your migration in the old Zope instance (where you have both the old "BTree" as the new "BTrees.XXBTree".
Then copy the storage over to the new Zope instance..
-- Dieter
____________________________________________________________________________________ Expecting? Get great news right away with email Auto-Check. Try the Yahoo! Mail Beta. http://advision.webevents.yahoo.com/mailbeta/newmail_tools.html
Ken Ara wrote at 2007-5-4 14:18 -0700:
... Could the solution be to redefine this product into something really simple and then kill it off?
There are receipes around how to get rid of objects that are not deletable by normal ways. Usually, they help to delete objects that cause "POSKeyError"s but they can also be used for otherwise "broken" objects. Of course, you would lose the content of these old "BTree" objects.
... * Module Products.BTreeFolder2.BTreeFolder2, line 368, in objectValues * Module Products.BTreeFolder2.BTreeFolder2, line 347, in objectIds
AttributeError: 'NoneType' object has no attribute 'keys'
This exception does not look as if caused by an old "BTree". Check for what object "keys" is called and ensure that it is not "None". I expect that the object is "_tree" and that it is "None" because you did not call the "BTreeFolder2.__init__". Call "BTreeFolder2.__init__" on the "BTreeFolder2" instance and this problem should go away. -- Dieter
Lawrence, As I ponder Dieter's solution I quickly tried yours... with the result as follows:
del app.OrgDocumentLibrary['Documents'] Traceback (most recent call last): File "<stdin>", line 1, in ? AttributeError: __delitem__
I'll get this eventually... Thanks Ken --- Laurence Rowe wrote:
Well, you're replicating what was going on in the
browser, so you got
the same results. Instead try:
del app.OrgDocumentLibrary['Documents']
You'll need to do a transaction.get().commit() to persist the changes.
--- Dieter Maurer wrote:
There are receipes around how to get rid of objects that are not deletable by normal ways. Usually, they help to delete objects that cause "POSKeyError"s but they can also be used for otherwise "broken" objects.
Of course, you would lose the content of these old "BTree" objects.
... * Module Products.BTreeFolder2.BTreeFolder2, line 368, in objectValues * Module Products.BTreeFolder2.BTreeFolder2, line 347, in objectIds
AttributeError: 'NoneType' object has no attribute 'keys'
This exception does not look as if caused by an old "BTree".
Check for what object "keys" is called and ensure that it is not "None".
I expect that the object is "_tree" and that it is "None" because you did not call the "BTreeFolder2.__init__". Call "BTreeFolder2.__init__" on the "BTreeFolder2" instance and this problem should go away.
__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
This kind of thing wasn't working:
del app.OrgDocumentLibrary['Documents'] Traceback (most recent call last): File "<stdin>", line 1, in ? AttributeError: __delitem__
so I tried an experiment on an empty catalog:
del app.OrgDocumentLibrary.Documents_catalog
No complaints from zopectl debug... Then
import transaction transaction.commit()
Except now I can't access the ZMI under OrgDocumentLibrary... Traceback (innermost last): * Module ZPublisher.Publish, line 119, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 42, in call_object * Module Shared.DC.Scripts.Bindings, line 313, in __call__ * Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec * Module App.special_dtml, line 178, in _exec * Module OFS.ObjectManager, line 426, in objectItems * Module OFS.ObjectManager, line 290, in _getOb AttributeError: Documents_catalog /// I can neither delete nor create an object called 'Documents_catalog'. any... ideas? Ken --- Dieter Maurer wrote:
This exception does not look as if caused by an old "BTree".
Check for what object "keys" is called and ensure that it is not "None".
I expect that the object is "_tree" and that it is "None" because you did not call the "BTreeFolder2.__init__". Call "BTreeFolder2.__init__" on the "BTreeFolder2" instance and this problem should go away.
-- Dieter
____________________________________________________________________________________ Get your own web address. Have a HUGE year through Yahoo! Small Business. http://smallbusiness.yahoo.com/domains/?p=BESTDEAL
Ken Ara wrote at 2007-5-7 13:47 -0700:
... objectItems * Module OFS.ObjectManager, line 290, in _getOb
AttributeError: Documents_catalog
"ObjectManager" has 2 references to its children: one in "_objects" and one as a direct attribute. You deleted the direct attribute but forget to delete the reference in "_objects", thus making the "ObjectManager" data structure inconsistent. The easiest way to fix this is something like: from OFS.DTMLMethod import DTMLMethod object_manager.Documents_catalog = DTMLMethod() object_manager._delObject('Documents_catalog') Then commit. This is also the easiest way to get rid of broken objects (as it avoids the inconsistencies in the first place). -- Dieter
This worked perfectly. Dieter, many thanks! I have fallen into a few pitfalls along the upgrade path. Now I understand why so many are still using Zope 2.7 and even earlier - it is scary. I tried to prepare for the upgrade but did not fully appreciate the risks. There are 'changes lists' but no compendium of gotchas for upgraders. By learning the use of zopectl debug I now understand that I need to fix code containing 'get_transaction'... And one important project dependent on Localizer is still broken. We are not out of the woods yet! Ken --- Dieter Maurer wrote:
Ken Ara wrote at 2007-5-7 13:47 -0700:
... objectItems * Module OFS.ObjectManager, line 290, in _getOb
AttributeError: Documents_catalog
"ObjectManager" has 2 references to its children: one in "_objects" and one as a direct attribute. You deleted the direct attribute but forget to delete the reference in "_objects", thus making the "ObjectManager" data structure inconsistent.
The easiest way to fix this is something like:
from OFS.DTMLMethod import DTMLMethod object_manager.Documents_catalog = DTMLMethod() object_manager._delObject('Documents_catalog')
Then commit.
This is also the easiest way to get rid of broken objects (as it avoids the inconsistencies in the first place).
-- Dieter
____________________________________________________________________________________ Be a PS3 game guru. Get your game face on with the latest PS3 news and previews at Yahoo! Games. http://videogames.yahoo.com/platform?platform=120121
Try deleting them with del from python (eg from zopectl debug) rather than through the ZMI. Laurence Ken Ara wrote:
I have a couple of outstanding issues following an otherwise successful upgrade from Zope 2.7.4 to 2.10.3-final.
One key application used the old DocumentLibrary product (http://www.zope.org/Members/Kaivo/DocumentLibrary). I have since reimplemented most of the functionality and migrated out the content, but I am unable to delete the old instances of the 'DocumentStore' class. This class is based on the old BTreeFolder which, as far as I can tell, cannot be be used in current versions of Zope.
Starting Zope with the old BTreeFolder fails:
Traceback (most recent call last): File "/usr/local/zope/lib/python/OFS/Application.py", line 703, in import_product product=__import__(pname, global_dict, global_dict, silly) File "/usr/local/zope/client0/Products/BTreeFolder/__init__.py", line 89, in ? import BTreeFolder File "/usr/local/zope/client0/Products/BTreeFolder/BTreeFolder.py", line 91, in ? from BTree import BTree ImportError: No module named BTree
Swapping in BTreeFolder2 allowed me to access existing instances of DocumentLibrary, however attempting to delete the old instances of DocumentStore fails with the following error:
Traceback (innermost last): Module ZPublisher.Publish, line 119, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 42, in call_object Module OFS.ObjectManager, line 524, in manage_delObjects Module OFS.ObjectManager, line 379, in _delObject Module zope.event, line 23, in notify Module zope.component.event, line 26, in dispatch Module zope.component._api, line 130, in subscribers Module zope.component.registry, line 290, in subscribers Module zope.interface.adapter, line 535, in subscribers Module zope.component.event, line 33, in objectEventNotify Module zope.component._api, line 130, in subscribers Module zope.component.registry, line 290, in subscribers Module zope.interface.adapter, line 535, in subscribers Module OFS.subscribers, line 108, in dispatchObjectWillBeMovedEvent Module zope.app.container.contained, line 181, in dispatchToSublocations Module OFS.subscribers, line 88, in sublocations Module Products.BTreeFolder2.BTreeFolder2, line 368, in objectValues Module Products.BTreeFolder2.BTreeFolder2, line 347, in objectIds AttributeError: 'NoneType' object has no attribute 'keys'
If I could install the original BTreeFolder in Zope 2.10.3 - if only briefly - would that enable me to delete the old instances? Other ideas?
Thanks, Ken
__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
participants (3)
-
Dieter Maurer -
Ken Ara -
Laurence Rowe