Hi all, Rather large message with tracebacks and all -- any help is greatly appreciated... I have tried to rid a large Zope DB of a POSKeyError to no avail. I have tried the method that traverses the folder which contains the offending object using try/except to catch the POSKeyError, but even those continually throw the error. Below is the traceback: ------------- Traceback (most recent call last): File "<stdin>", line 1, in ? File "error_finder.py", line 6, in error_finder for id, next_item in folder.objectItems(): File "/home/zope/instances/primary/Products/BTreeFolder2/BTreeFolder2.py", line 369, in objectItems self.objectIds(spec)) File "/opt/Zope-2.7/lib/python/Products/ZCatalog/Lazy.py", line 138, in __init__ else: self._len = len(seq) File "/opt/Zope-2.7/lib/python/ZODB/Connection.py", line 559, in setstate p, serial = self._storage.load(oid, self._version) File "/opt/Zope-2.7/lib/python/ZODB/FileStorage.py", line 689, in load return self._load(oid, version, self._index, self._file) File "/opt/Zope-2.7/lib/python/ZODB/FileStorage.py", line 657, in _load raise POSKeyError(oid) ZODB.POSException.POSKeyError: 00000000000e5fec -------------------- I also ran fsrefs.py and the output is below: -------------------- oid 0x47152L BTrees.OOBTree.OOBucket last updated: 2005-04-19 22:45:48.466296, tid=0x35CB635CECA1E99L refers to invalid object: oid 0xD5B62L missing: 'Products.CMFDefault.File.File' oid 0x25506L BTrees.OOBTree.OOBucket last updated: 2005-09-19 10:00:59.079910, tid=0x3601B18FC130433L refers to invalid objects: oid 0xE5FEBL missing: 'Products.CMFDefault.File.File' oid 0xE5FECL missing: 'BTrees.OOBTree.OOBucket' oid 0x393C8L BTrees.OOBTree.OOBucket last updated: 2005-06-28 08:05:37.236789, tid=0x35E42259EE080DDL refers to invalid object: oid 0xDB6C5L missing: 'Products.CMFDefault.File.File' oid 0x18235L BTrees.OOBTree.OOBucket last updated: 2005-09-03 05:26:28.090544, tid=0x35FC00677DA5D66L refers to invalid object: oid 0xD2F9BL missing: 'Products.CMFDefault.File.File' oid 0xE43BEL BTrees.OOBTree.OOBucket last updated: 2005-08-16 06:58:59.474963, tid=0x35F5B22FDC28500L refers to invalid objects: oid 0xE2A83L missing: 'Products.CMFDefault.File.File' oid 0xE2AADL missing: 'Products.CMFDefault.File.File' oid 0xE2AB8L missing: 'Products.CMFDefault.File.File' oid 0xE4FAEL BTrees.OOBTree.OOBucket last updated: 2005-08-29 04:59:25.040228, tid=0x35FA3CB6AD69B33L refers to invalid objects: oid 0xE19C4L missing: 'BTrees.IOBTree.IOBTree' oid 0xE19CDL missing: 'BTrees.IOBTree.IOBTree' oid 0xE19D7L missing: 'BTrees.IOBTree.IOBTree' ---------------- All of the objects that are causing the POSKeyError I believe to be coming from a single instance of a very large BTreeFolder2 instance. Can anyone offer any remedy to remove or repair the offending objects? Thanks, Kevin
Kevin Carlson wrote at 2005-9-24 22:28 -0400:
... File "/opt/Zope-2.7/lib/python/Products/ZCatalog/Lazy.py", line 138, in __init__ else: self._len = len(seq) File "/opt/Zope-2.7/lib/python/ZODB/Connection.py", line 559, in setstate p, serial = self._storage.load(oid, self._version) File "/opt/Zope-2.7/lib/python/ZODB/FileStorage.py", line 689, in load return self._load(oid, version, self._index, self._file) File "/opt/Zope-2.7/lib/python/ZODB/FileStorage.py", line 657, in _load raise POSKeyError(oid) ZODB.POSException.POSKeyError: 00000000000e5fec
Your catalog contains the missing object
oid 0x25506L BTrees.OOBTree.OOBucket last updated: 2005-09-19 10:00:59.079910, tid=0x3601B18FC130433L refers to invalid objects: oid 0xE5FEBL missing: 'Products.CMFDefault.File.File' oid 0xE5FECL missing: 'BTrees.OOBTree.OOBucket'
The second is the one shown in your traceback above. This means: the "OOBucket" with oid "0x25506" contains two broken references. Load it from the ZODB. Determine which keys belong to the broken references and delete these keys.
... All of the objects that are causing the POSKeyError I believe to be coming from a single instance of a very large BTreeFolder2 instance. Can anyone offer any remedy to remove or repair the offending objects?
You can fix them all as shown above. While the "POSKeyError"s will go away, you have (of course) lost information. -- Dieter
Dieter, I'm not sure I understand what you mean by fixing them all as shown above. You mention loading the object from the ZODB and deleting the broken references, but I'm not sure how to do that. I have looked around for info on loading objects by oid but cannot find anything helpful. Can you help me out? Thanks, Kevin Dieter Maurer wrote:
Kevin Carlson wrote at 2005-9-24 22:28 -0400:
... File "/opt/Zope-2.7/lib/python/Products/ZCatalog/Lazy.py", line 138, in __init__ else: self._len = len(seq) File "/opt/Zope-2.7/lib/python/ZODB/Connection.py", line 559, in setstate p, serial = self._storage.load(oid, self._version) File "/opt/Zope-2.7/lib/python/ZODB/FileStorage.py", line 689, in load return self._load(oid, version, self._index, self._file) File "/opt/Zope-2.7/lib/python/ZODB/FileStorage.py", line 657, in _load raise POSKeyError(oid) ZODB.POSException.POSKeyError: 00000000000e5fec
Your catalog contains the missing object
oid 0x25506L BTrees.OOBTree.OOBucket last updated: 2005-09-19 10:00:59.079910, tid=0x3601B18FC130433L refers to invalid objects: oid 0xE5FEBL missing: 'Products.CMFDefault.File.File' oid 0xE5FECL missing: 'BTrees.OOBTree.OOBucket'
The second is the one shown in your traceback above.
This means: the "OOBucket" with oid "0x25506" contains two broken references.
Load it from the ZODB. Determine which keys belong to the broken references and delete these keys.
... All of the objects that are causing the POSKeyError I believe to be coming from a single instance of a very large BTreeFolder2 instance. Can anyone offer any remedy to remove or repair the offending objects?
You can fix them all as shown above.
While the "POSKeyError"s will go away, you have (of course) lost information.
Kevin Carlson wrote at 2005-10-10 18:37 -0400:
... I'm not sure I understand what you mean by fixing them all as shown above. You mention loading the object from the ZODB and deleting the broken references, but I'm not sure how to do that.
"fsrefs" tells you the OID of objects containing broken references. When you have the OID (as binary value; I think "fsrefs" tells you the hex value, then can use "hexvalue.decode('hex')" to get it as binary), you can load the respective object via obj = connection[OID] You get the connection from an object via "obj._p_jar". In your case, an "OOBucket" lost some of its references (that is an easy case -- provided it does not loose all its content). You determine the broken keys and delete them: for k,v in list(bucket.objectItems()): try: v._p_changed = 0 # load the object except: # broken del bucket[k] The case becomes very complicated, when the bucket ends empty -- an empty bucket would make the (probably) enclosing BTree invalid. You would need to fake something to keep it non-empty. if not bucket: bucket[k] = None # "k" was one of the deleted keys -- Dieter
participants (2)
-
Dieter Maurer -
Kevin Carlson