[ZODB-Dev] Fixing POSKeyErrors :-)

[Chris Withers]

Okay, so I found a fix for my problem, thought I'd share for others.

So, you have an error like this:

<blah>
POSKeyError: 0x3e0a

...do a bin/zopectl debug and then do roughly the following:

 >>> from ZODB.utils import p64
 >>> oid = p64(0x3e0a)

So now we've got the oid of the broken object, lets create something to 
replace it with:

 >>> from persistent.mapping import PersistentMapping
 >>> data = PersistentMapping()
 >>> data['BROKEN']=True

It's important that this is easy to spot:

 >>> repr(data)
"{'BROKEN': True}"

...'cos once it's fixed, you'll need to look for that dict-ish thing in 
various attempts of print(vars(object_that_gave_you_the_POSKeyError)) 
until you've got rid of all instances of it.

Anyway, lets get on and insert our 'broken' marker so that we don't get 
POSKeyErrors raised:

 >>> import cStringIO
 >>> import cPickle
 >>> import transaction
 >>> s = app._p_jar.db()._storage
 >>> file = cStringIO.StringIO()
 >>> p = cPickle.Pickler(file, 1)
 >>> p.dump((data.__class__, None))
<cPickle.Pickler object at 0xb5eb1134>
 >>> p.dump(data.__getstate__())
<cPickle.Pickler object at 0xb5eb1134>
 >>> t = transaction.Transaction()
 >>> t.description = 'Fix POSKeyError'
 >>> s.tpc_begin(t)
 >>> s.store(oid,None,file.getvalue(),'',t)
 >>> s.tpc_vote(t)
[('\x00\x00\x00\x00\x00\x00>\n', '\x03s\xb6\xf4AQ\xe7\xdd')]
 >>> s.tpc_finish(t)

Okay, that's the hairy bit over, now you need to do 'the usual' things 
to remove or fix things that refer to the new data.

Does anyone know where the best place is to put How-To's of this sort 
nowadays?

It's a shame ZODB doesn't turn POSKeyErrors into proper Broken objects 
as it does when the class can no longer be imported. The problem with 
POSKeyErrors is that they prevent you accessing *any object that refers 
to the broken object* not just the missing object itself. This means 
that when objects *do* go missing, the data loss can be much much worse 
than it needs to be.

What does everyone else think?

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk