Hi all, Whenever I try to reindex an index in the portal_catalog or delete the index, add it back and then reindex, I get the first error below. I ran fstest.py and there were no errors in the Data.fs file yet. I also ran fsrecover.py and there were zero bytes removed from the Data.fs file. Also, When I run fsrefs.py, I get the second bit of output below. You may have seen my recent posts about the BTreeFolder2 instance that stores the vast majority of the content for this application. It has been suggested that the BTreeFolder2 instance may have corrupt BTrees so I rebuilt those using Tres Seaver's method but the behavior stays the same. Any ideas on this? Exception Type POSKeyError Exception Value 00000000000cc49a Traceback (innermost last): * Module ZPublisher.Publish, line 101, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module Products.ZCatalog.ZCatalog, line 499, in manage_reindexIndex * Module Products.ZCatalog.ZCatalog, line 485, in reindexIndex * Module Products.CMFCore.CatalogTool, line 207, in catalog_object * Module Products.CMFCore.WorkflowTool, line 353, in getCatalogVariablesFor * Module Products.CMFCore.WorkflowTool, line 338, in getWorkflowsFor * Module Products.CMFCore.WorkflowTool, line 307, in getChainFor * Module ZODB.Connection, line 559, in setstate * Module ZODB.FileStorage, line 689, in load * Module ZODB.FileStorage, line 657, in _load POSKeyError: 00000000000cc49a -------------------------------- fsrefs.py output oid 0xC8DF8L BTrees.OOBTree.OOBucket last updated: 2005-01-25 09:42:25.200913, tid=0x35AC9C66B861E00L refers to invalid object: oid 0xCC49AL missing: 'Products.CMFDefault.File.File' oid 0xC8E1FL BTrees.OOBTree.OOBucket last updated: 2005-01-25 09:42:25.200913, tid=0x35AC9C66B861E00L refers to invalid object: oid 0xCC49CL missing: 'BTrees.IOBTree.IOBTree' oid 0xD297EL BTrees.OOBTree.OOBucket last updated: 2005-02-10 04:08:47.313263, tid=0x35B2278C9DEB333L refers to invalid object: oid 0xCC49AL missing: 'Products.CMFDefault.File.File'
[Kevin Carlson]
Whenever I try to reindex an index in the portal_catalog or delete the index, add it back and then reindex, I get the first error below. I ran fstest.py and there were no errors in the Data.fs file yet. I also ran fsrecover.py and there were zero bytes removed from the Data.fs file.
Which means the low-level structure of your Data.fs file is fine; roughly speaking, it's like a sentence that's grammatical but doesn't completely make sense. Please read this for background info; you'll find it worth your time: http://zope.org/Wikis/ZODB/FileStorageBackup If fstest.py has no complaints, it's unlikely fsrecover.py will change anything. Those two go together.
Also, When I run fsrefs.py, I get the second bit of output below.
Which I'm moving up to here:
fsrefs.py output
oid 0xC8DF8L BTrees.OOBTree.OOBucket last updated: 2005-01-25 09:42:25.200913, tid=0x35AC9C66B861E00L refers to invalid object: oid 0xCC49AL missing: 'Products.CMFDefault.File.File'
oid 0xC8E1FL BTrees.OOBTree.OOBucket last updated: 2005-01-25 09:42:25.200913, tid=0x35AC9C66B861E00L refers to invalid object: oid 0xCC49CL missing: 'BTrees.IOBTree.IOBTree'
oid 0xD297EL BTrees.OOBTree.OOBucket last updated: 2005-02-10 04:08:47.313263, tid=0x35B2278C9DEB333L refers to invalid object: oid 0xCC49AL missing: 'Products.CMFDefault.File.File'
So two objects have vanished from this database, with object ids (oids) 0xCC49A and 0xCC49C, and in total they're referenced _from_ three other objects. As a result, any attempt to reference oids 0xCC49A or 0xCC49C will raise POSKeyError -- it's akin to the KeyError Python raises if you try to look up a key that doesn't exist in a Python dict. Here a few objects refer to other objects that don't actually exist. There's isn't a wholly safe automated way to "fix that". ...
Exception Type POSKeyError Exception Value 00000000000cc49a
Note that oid 0xcc49a was identified by fsrefs.py as one of the two oids (the missing object of type Products.CMFDefault.File.File) that just don't exist in this database. One way or another, the database has to be changed to stop referring to this (non-existent) object, else you'll continue to get a POSKeyError whenever an attempt to reference it is made. fsrecover.py can't do anything about this (which should be clearer if you read the page linked to above). The script mentioned in this recent message may (or may not) be useful to you: http://mail.zope.org/pipermail/zodb-dev/2005-January/008415.html Note that, as it says, running it is guaranteed to cause data loss -- the resulting Data.fs may or may not be usable (it should be well-formed (fstest.py shouldn't have any complaints), and fsrefs.py should stop complaining too, but the script may delete objects containing data you care about). BTW, which version of Zope are you using here? Major fixes went into Zope 2.7.3 + ZODB 3.2.4 to repair bugs that could cooperate to create POSKeyError situations (both transient and permanent). Zope 2.7.4 + ZODB 3.2.5 are the current stable releases. There are no known Zope or ZODB bugs that can cause POSKeyError situations in these recent versions.
participants (2)
-
Kevin Carlson -
Tim Peters