RH7.2 Zope 2.7.2 ZEO Squid in front
~ What version of Zope? You may need to run 'fstest.py' and / or ~ 'fsrefs.py' on your Data.fs, and see if they complain. I am assuming these are just reading tools and non-destructive? Can they be run against a running zeo/zope?
Short course: yes; yes; if you must but it's not recommended (won't hurt but you may get spurious error messages then). Medium course here:
http://zope.org/Wikis/ZODB/FileStorageBackup
You should read that. POSKeyError "shouldn't happen", so there's no guarantee it will be easy to fix. Unfortunately, in Zope 2.7.2 there were cooperating bugs (in ZODB and Zope) that could create POSKeyError situations through no fault of yours; those were repaired in Zope 2.7.3 / ZODB 3.2.4.
Your original message didn't contain anything from any log, or copy'n'paste from any browser window, that said "POSKeyError". I don't think it's possible to get a POSKeyError without the message also including vital information like the object id of the missing object -- but we didn't see the message you saw, so can't tell.
My bad... The POSKeyError showed up on the browser window as a result of running the "/manage_addProduct/PythonScripts/recompile" script. Zope Error Zope has encountered an error while publishing this resource. Error Type: POSKeyError Error Value: 00000000008db667
From error_log:
Traceback (innermost last): Module ZPublisher.Publish, line 101, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 39, in call_object Module Products.PythonScripts, line 53, in recompile Module OFS.FindSupport, line 151, in ZopeFind Module OFS.FindSupport, line 151, in ZopeFind Module OFS.FindSupport, line 151, in ZopeFind Module OFS.FindSupport, line 138, in ZopeFind Module ZODB.Connection, line 559, in setstate Module ZEO.ClientStorage, line 749, in load Module ZEO.ServerStub, line 82, in zeoLoad Module ZEO.zrpc.connection, line 372, in call POSKeyError: 00000000008db667