[ZODB-Dev] POSKeyError

[Tim Peters]

[Bob Horvath]
> I have a working site that has several domains all in the same database
> file, one of which has POSKeyErrors. I have tried the various recipes,
> but I am very concerned I'll screw things up without a better
> understanding.

If you haven't read this, you should:

    http://zope.org/Wikis/ZODB/FileStorageBackup

> As I understand it, POSKeyErrors are either objects that have no path
> from the root, or objects that refer to other objects that don't exist.

That's too complicated.  Each object has an identifier, called an oid.  A
storage is essentially a mapping from oids to objects.  POSKeyError occurs
when a storage is asked to fetch the object associated with an oid, but the
storage doesn't know about that oid.  It's very similar to the KeyError you
get when you index a Python dictionary by a key that isn't in the
dictionary.

There are really two immediate causes for POSKeyError:

1. The oid simply isn't in the database.

2. The oid is in the database, but the transaction that created
   the object was undone.  Then the object still exists, but only
   in the past, and the object could become visible again if
   the undo of its creation were itself undone.

In practice, #2 is at work if and only if "_loadBack_impl" shows up in a
POSKeyError traceback.

> For the first case, deleting them seems to be harmless,

An object in the database not reachable from the root isn't a cause of
POSKeyError.  It's normal that objects become unreachable over time.  For
FileStorage, packing gets rid of objects, and object revisions, that are no
longer reachable from the root.

> but for the second, if you have a folder of things one of which doesn't
> exist, deleting the folder seems more destructive than necessary.
> Wouldn't it be better to delete the reference to the item?

Probably.  No tool automates that, though.  You can use fsrefs.py to find
"dangling references" in a FileStorage.  The version of fsrefs.py in ZODB
3.2.4c1 (the most recent 3.2 ZODB package you can download from

    http://www.zope.org/Products/ZODB3.2

) is the best version of that tool available.

> If I am using the scripts right, these are the items with errors...

It's unclear which scripts you're talking about, and what "with errors"
means exactly.

> 'acl_users'
> 'Control_Panel'
> 'standard_html_header'
> 'standard_html_footer'
> 'standard_error_message'
> 'index_html'
> 'temp_folder'
> 'browser_id_manager'
> 'session_data_manager'
> 'method_id'
> 'error_log'
>
> These would seem scary to just delete. I have been in this state for
> months and have not been able to fix it.  Any suggestions?

Sorry, I don't have specific advice.  General <wink> advice is to upgrade to
Zope 2.7.3 (you didn't say which version of Zope or ZODB you're using) as
soon as it's released.  POSKeyErrors are supposed to be "impossible", but at
least Zope 2.7.2 and ZODB conspired to create them in some cases, and those
bugs will be fixed in 2.7.3.