Hi there, There seems to be a problem with my ZCatalog, and it seems to be getting worse as time goes by. When my CatalogPathAware objects unindex themselves, sometimes I get an attribute error like: AttributeError: 'str' object has no attribute 'append' Traceback (innermost last): Module ZPublisher.Publish, line 101, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 39, in call_object Module Products.FileSystemSite.FSPythonScript, line 104, in __call__ Module Shared.DC.Scripts.Bindings, line 306, in __call__ Module Shared.DC.Scripts.Bindings, line 343, in _bindAndExec Module Products.FileSystemSite.FSPythonScript, line 160, in _exec Module None, line 33, in tab_status_approve - <FSPythonScript at /silva/service_views/Silva/edit/VersionedContent/tab_status_approve used for /silva/service_views/SilvaDocument/edit/VersionedContent/Document> - Line 33 Module Products.Silva.Versioning, line 125, in approve_version Module Products.Silva.Versioning, line 512, in _update_publication_status Module Products.Silva.VersionedContent, line 334, in _unindex_version Module Products.Silva.Version, line 191, in unindex_object Module Products.ZCatalog.ZCatalog, line 558, in uncatalog_object Module Products.ZCatalog.Catalog, line 411, in uncatalogObject Module Products.ZCTextIndex.ZCTextIndex, line 181, in unindex_object Module Products.ZCTextIndex.OkapiIndex, line 68, in unindex_doc Module Products.ZCTextIndex.BaseIndex, line 176, in unindex_doc Module Products.ZCTextIndex.BaseIndex, line 303, in _del_wordinfo Module ZODB.Connection, line 562, in setstate Module ZODB.Connection, line 601, in _set_ghost_state AttributeError: 'str' object has no attribute 'append' In the logfile, the last line of code is shown in the traceback: File "/export1/Zope-2.7.2-0/lib/python/ZODB/Connection.py", line 601, in _set_ghost_state state = unpickler.load() Perhaps this is a ZEO issue. I'm using Zope-2.7.2-0 with ZEO. Both the zeo server and the client are on the same box. The zeo logfile reports no errors when this occurs. Any ideas why this may be happening?
I just began receiving this error this morning in one of my event logs on start up. It looks serious. 2005-08-08T08:49:26 WARNING ZODB.FileStorage Failed to load database index: exceptions.AttributeError: type object 'BTrees._fsBTree.fsBTree' has no attribute '__basicnew__' Can someone advise a course of action and what may have happened to generate such an error. I use CMFBTreeFolders extensively in CMF. Many thanks. Regards, David
Hmm... not sure what causes this (did you upgrade ZODB versions?) but you may try: 1. Shut down Zope (or ZEO if you use that). 2. find your Data.fs file 3. Delete the file beside it named Data.fs.index 4. Restart Zope (or ZEO). See if the error goes away.. FileStorage builds an index of oid to file position in this index. It's transient, and will be rebuilt when the process is restarted. - C On Mon, 2005-08-08 at 11:11 -0300, David Pratt wrote:
I just began receiving this error this morning in one of my event logs on start up. It looks serious.
2005-08-08T08:49:26 WARNING ZODB.FileStorage Failed to load database index: exceptions.AttributeError: type object 'BTrees._fsBTree.fsBTree' has no attribute '__basicnew__'
Can someone advise a course of action and what may have happened to generate such an error. I use CMFBTreeFolders extensively in CMF. Many thanks.
Regards, David _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Hi Chris. Thanks for your reply. It is a relatively new ZODB from a fresh 2.8.0 installed a week or so ago. I went back over what I had been doing last evening with the site and it looks like something may have happened with a CMFBTree Folder instance - not sure exactly what at this point since there was no traceback. I am adding nested CMFBTreeFolders (3 levels) as part of a product using manage_addProduct a statements like the following. Is there anything wrong with this approach? btree_url0_folder_obj.manage_addProduct['BTreeFolder2'].manage_addConten t(type='CMF BTree Folder', id=btree_url1_folder_id) I removed the last instance I created last night and restarted and everything looks good again. Not sure it there is some sort of bug lurking but will be keep a close watch for any repeat of this in future. Thanks David
David Pratt wrote at 2005-8-8 11:11 -0300:
I just began receiving this error this morning in one of my event logs on start up. It looks serious.
2005-08-08T08:49:26 WARNING ZODB.FileStorage Failed to load database index: exceptions.AttributeError: type object 'BTrees._fsBTree.fsBTree' has no attribute '__basicnew__'
Can someone advise a course of action and what may have happened to generate such an error. I use CMFBTreeFolders extensively in CMF. Many thanks.
Mysterious: "BTrees._fsBTree.fsBTree" should have a '__basicnew__' attribute (like all 'ExtensionClass'es). Try to import "fsBTree" in an interactive Python interpreter and check it. -- Dieter
Hi Dieter. It is rather strange for sure. The warning has appeared over the past couple of days. First day I removed what I thought might be a bad CMFBTree folder since it was last thing I remember working with from previous night to do with BTrees. Then it appeared again - first restart of the day - it never appeared again that day even after a number of restarts. Received the error again this morning - first restart of day. This time cleared the Data.fs.index and restarted. Again it did not appear again all day after a number of restarts. It almost makes me wonder if it is somehow related to automated backup since it appears consistently on the first restart of the day. Can this make any sense? Like the index is getting corrupted from backup. The only other thing that I can think of at the moment is that I had some nested CMFBTreeFolders that were not registered properly registered in the types tool yet inside a CMFSite instance. These were added programmatically with manage_add. I don't know if this could result in this error. In any case, I am resolving the types tool registration issue tonight and will see what happens tomorrow am. I am restarting this zope several times a day since I am developing on it. I have been adding and removing many BTree folders since I am in process of building a product that will use these quite a bit. I have been running tests then removing CMFBTreeFolders, updating product, restarting, rerunning with changes, etc In any case I will examine fsBtree as you have suggested if I see warning again tomorrow in my log. This is a relatively new CMF1.5.2 and Zope2.8.0 instance and the only server I have this combination on at the moment. Regards, David On Wednesday, August 10, 2005, at 06:37 PM, Dieter Maurer wrote:
David Pratt wrote at 2005-8-8 11:11 -0300:
I just began receiving this error this morning in one of my event logs on start up. It looks serious.
2005-08-08T08:49:26 WARNING ZODB.FileStorage Failed to load database index: exceptions.AttributeError: type object 'BTrees._fsBTree.fsBTree' has no attribute '__basicnew__'
Can someone advise a course of action and what may have happened to generate such an error. I use CMFBTreeFolders extensively in CMF. Many thanks.
Mysterious:
"BTrees._fsBTree.fsBTree" should have a '__basicnew__' attribute (like all 'ExtensionClass'es).
Try to import "fsBTree" in an interactive Python interpreter and check it.
-- Dieter
David Pratt wrote at 2005-8-10 22:07 -0300:
Hi Dieter. It is rather strange for sure. The warning has appeared over the past couple of days. First day I removed what I thought might be a bad CMFBTree folder since it was last thing I remember working with from previous night to do with BTrees. Then it appeared again - first restart of the day - it never appeared again that day even after a number of restarts. Received the error again this morning - first restart of day. This time cleared the Data.fs.index and restarted. Again it did not appear again all day after a number of restarts. It almost makes me wonder if it is somehow related to automated backup since it appears consistently on the first restart of the day. Can this make any sense?
No, it does not.... But nevertheless, you see what you see? "__basicnew__" is a class method. It must be there independent on the concrete object. If it were not there, then whether or not you see the error may depend on whether or not the index file is present (you should not see it when there is no index file; you may, if there is). But, it should be there. If not, then the shared object "BTrees/_fsBTree.{so,pyd}" must have been badly executed (maybe because it is corrupt) -- but it would be surprising that this shows up in such a harmless way (one would instead expect a memory violation or bad opcode or something really nasty). -- Dieter
follow-up from my post earlier today: I ran fsrecover.py on a copy of my Data.fs. It found no errors, but then when it attempted to pack the file, it raised an unpickleable error, with this traceback: Traceback (most recent call last): File "fsrecover.py", line 374, in ? main() File "fsrecover.py", line 243, in main recover(inp, outp, verbose, partial, force, pack) File "fsrecover.py", line 369, in recover ofs.pack(pack, referencesf) File "/export1/Zope-2.7.5-final/lib/python/ZODB/FileStorage.py", line 1582, in pack opos = p.pack() File "/export1/Zope-2.7.5-final/lib/python/ZODB/fspack.py", line 700, in pack self.gc.findReachable() File "/export1/Zope-2.7.5-final/lib/python/ZODB/fspack.py", line 456, in findReachable self.findReachableAtPacktime([z64]) File "/export1/Zope-2.7.5-final/lib/python/ZODB/fspack.py", line 531, in findReachableAtPacktime todo.extend(self.findrefs(pos)) File "/export1/Zope-2.7.5-final/lib/python/ZODB/fspack.py", line 604, in findrefs return referencesf(self._file.read(dh.plen)) File "/export1/Zope-2.7.5-final/lib/python/ZODB/referencesf.py", line 38, in referencesf raise ValueError, 'Error unpickling, %s' % p ValueError: Error unpickling, and the rest is presumably the pickle. I receive this same error when trying to pack using the ZMI. Any idea what's going on? Thanks for the help, Andy On Monday 08 August 2005 08:15 am, Andy Altepeter wrote:
Hi there,
There seems to be a problem with my ZCatalog, and it seems to be getting worse as time goes by. When my CatalogPathAware objects unindex themselves, sometimes I get an attribute error like:
AttributeError: 'str' object has no attribute 'append'
Traceback (innermost last): Module ZPublisher.Publish, line 101, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 39, in call_object Module Products.FileSystemSite.FSPythonScript, line 104, in __call__ Module Shared.DC.Scripts.Bindings, line 306, in __call__ Module Shared.DC.Scripts.Bindings, line 343, in _bindAndExec Module Products.FileSystemSite.FSPythonScript, line 160, in _exec Module None, line 33, in tab_status_approve - <FSPythonScript at /silva/service_views/Silva/edit/VersionedContent/tab_status_approve used for /silva/service_views/SilvaDocument/edit/VersionedContent/Document> - Line 33 Module Products.Silva.Versioning, line 125, in approve_version Module Products.Silva.Versioning, line 512, in _update_publication_status Module Products.Silva.VersionedContent, line 334, in _unindex_version Module Products.Silva.Version, line 191, in unindex_object Module Products.ZCatalog.ZCatalog, line 558, in uncatalog_object Module Products.ZCatalog.Catalog, line 411, in uncatalogObject Module Products.ZCTextIndex.ZCTextIndex, line 181, in unindex_object Module Products.ZCTextIndex.OkapiIndex, line 68, in unindex_doc Module Products.ZCTextIndex.BaseIndex, line 176, in unindex_doc Module Products.ZCTextIndex.BaseIndex, line 303, in _del_wordinfo Module ZODB.Connection, line 562, in setstate Module ZODB.Connection, line 601, in _set_ghost_state AttributeError: 'str' object has no attribute 'append'
In the logfile, the last line of code is shown in the traceback: File "/export1/Zope-2.7.2-0/lib/python/ZODB/Connection.py", line 601, in _set_ghost_state state = unpickler.load()
Perhaps this is a ZEO issue. I'm using Zope-2.7.2-0 with ZEO. Both the zeo server and the client are on the same box. The zeo logfile reports no errors when this occurs.
Any ideas why this may be happening? _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Andy Altepeter]
I ran fsrecover.py on a copy of my Data.fs. It found no errors, but then when it attempted to pack the file, it raised an unpickleable error, with this traceback: ... File "/export1/Zope-2.7.5-final/lib/python/ZODB/referencesf.py", line 38, in referencesf raise ValueError, 'Error unpickling, %s' % p ValueError: Error unpickling, and the rest is presumably the pickle.
I receive this same error when trying to pack using the ZMI. Any idea what's going on?
It suggests data corruption in your Data.fs file (assuming you're using FileStorage). Please read this for background info: http://zope.org/Wikis/ZODB/FileStorageBackup Because fsrecover.py completed without error, best guess is that fstest.py (described in the link above) would also complete without error. However, neither of those can detect "higher-level" corruption inside serialized object state (see the link for more words). Running fsrefs.py (see the link again) can identify most such corruption. So run it and see what it says. There is no wholly safe and automatic way to recover from frsrefs.py-level corruption (short of reverting to a good backup), so don't expect this to go easily. Note that better versions of these tools exist in more recent versions of ZODB (IIRC, you said you were running Zope-2.7.2, which corresponds to ZODB 3.2.3 -- ZODB 3.2.9 is the current production 3.2 release).
Hi Tim, Thank you for your response. fsrefs.py raised the same error I reported earlier about unpickling. I ended up: 1) Creating a new instance_home using zope-2.5.0-final 2) using zopectl debug connected to the bad database, I exported all my objects. For folders that couldn't be exported, I started 'drilling down' until I found the object that was causing problems. There were about 5 such objects in my database. - Some objects (folderish) could be exported, but then would raise a POSKey error upon import, so I drilled down in those folders as well. 3) I imported each .zexp into the new database. This process was time consuming, but it appears to have recreated the ZODB in a sane state. I think it's time to look into using repozo for backups ;-) Andy On Monday 08 August 2005 05:45 pm, Tim Peters wrote:
[Andy Altepeter]
I ran fsrecover.py on a copy of my Data.fs. It found no errors, but then when it attempted to pack the file, it raised an unpickleable error, with this traceback:
...
File "/export1/Zope-2.7.5-final/lib/python/ZODB/referencesf.py", line 38, in referencesf raise ValueError, 'Error unpickling, %s' % p ValueError: Error unpickling, and the rest is presumably the pickle.
I receive this same error when trying to pack using the ZMI. Any idea what's going on?
It suggests data corruption in your Data.fs file (assuming you're using FileStorage). Please read this for background info:
http://zope.org/Wikis/ZODB/FileStorageBackup
Because fsrecover.py completed without error, best guess is that fstest.py (described in the link above) would also complete without error. However, neither of those can detect "higher-level" corruption inside serialized object state (see the link for more words). Running fsrefs.py (see the link again) can identify most such corruption. So run it and see what it says. There is no wholly safe and automatic way to recover from frsrefs.py-level corruption (short of reverting to a good backup), so don't expect this to go easily.
Note that better versions of these tools exist in more recent versions of ZODB (IIRC, you said you were running Zope-2.7.2, which corresponds to ZODB 3.2.3 -- ZODB 3.2.9 is the current production 3.2 release).
Andy Altepeter wrote at 2005-8-8 08:15 -0500:
There seems to be a problem with my ZCatalog, and it seems to be getting worse as time goes by. ... Module ZODB.Connection, line 562, in setstate Module ZODB.Connection, line 601, in _set_ghost_state AttributeError: 'str' object has no attribute 'append'
In fact, this looks like a broken pickle in your ZODB storage (and not like a ZCatalog problem). -- Dieter
On Wednesday 10 August 2005 04:32 pm, Dieter Maurer wrote:
Andy Altepeter wrote at 2005-8-8 08:15 -0500:
There seems to be a problem with my ZCatalog, and it seems to be getting worse as time goes by. ... Module ZODB.Connection, line 562, in setstate Module ZODB.Connection, line 601, in _set_ghost_state AttributeError: 'str' object has no attribute 'append'
In fact, this looks like a broken pickle in your ZODB storage (and not like a ZCatalog problem).
I think I've fixed this issue, but for future reference, is there any way to remove a broken pickle?
Andy Altepeter wrote at 2005-8-11 07:22 -0500:
...
In fact, this looks like a broken pickle in your ZODB storage (and not like a ZCatalog problem).
I think I've fixed this issue, but for future reference, is there any way to remove a broken pickle?
No, because there should be no broken pickle in the first place... ...and because the problem did not yet occured often enough... -- Dieter
participants (5)
-
Andy Altepeter -
Chris McDonough -
David Pratt -
Dieter Maurer -
Tim Peters