[Derek J. Balling]
I'm resending this because a couple weeks later and not a single peep was to be heard, and my boss is starting to hound me for insight. :-(
Anyone have any helpful advice?
If your boss is willing to pay someone to help, that opens other possibilities. Otherwise this has to change:
K, now I don't mean to sound stupid, but I inherited a Zope install, and I know absolutely nothing about: (a) python (b) Zope (c) the application we run on this platform (MailManager)
That is, you'll have to teach yourself. Unfortunately, you're jumping in at the intersection of several deep ends: you have a corrupted Data.fs file, you don't have backups, and the problem first occurred a long time ago. We know that with high probability because:
If I run fstest.py I get:
# python /usr/local/Zope-2.7.0/utilities/ZODBTools/fstest.py /home/dballing/Data.fs /home/dballing/Data.fs has inconsistent transaction length for undone transaction at 475866827
If you read: http://zope.org/Wikis/ZODB/FileStorageBackup then you already know that .fs files grow only by appending, and your file is corrupt starting near byte 475,866,827. You originally said:
... Since the database is up to 5.X GB ...
so the damage occurred 4 or 5 gigabytes' worth of transactions in the past, or some process sprayed bytes to random spots on your disk (e.g., flaky disk controllers can exhibit that symptom, esp. under high load). When fstest.py fails, there's no (well, little) point to running fsrefs.py. It's possible you can recover by: 1. Bring your app down. 2. Make a physical copy of your Data.fs. 3. Learn how to use fsrecover.py, and try it on the copy. fstest.py should pass after that. fsrefs.py may or may not. There's no guarantee it will work: fsrecover.py proceeds simply by throwing away corrupted pieces of the FileStorage -- it's a very low-level program, with no semantic knowledge, so can create a well-formed file at a low level that's nevertheless missing objects crucial to your application. There's simply no magical fix for file corruption, whether it's a FileStorage or your email folder. It's possible (as Tres suggested) that someone reading the zodb-dev list would be able to volunteer more significant help than you're getting here.