[Fwd: Re: [Zope] Pack Not Working]
I'm resending this because a couple weeks later and not a single peep was to be heard, and my boss is starting to hound me for insight. :-( Anyone have any helpful advice? D -------- Original Message -------- Subject: Re: [Zope] Pack Not Working Date: Tue, 16 Nov 2004 15:28:25 -0500 From: Derek J. Balling <dballing@dpsource.com> Organization: Decorative Product Source, Inc. To: zope@zope.org References: <419A58E2.5070208@dpsource.com> <1f7befae04111611596a0e92bb@mail.gmail.com> Tim Peters wrote:
Next step is to read this:
http://zope.org/Wikis/ZODB/FileStorageBackup
then start investigating the nature of the damage with fstest.py and fsrefs.py.
K, now I don't mean to sound stupid, but I inherited a Zope install, and I know absolutely nothing about: (a) python (b) Zope (c) the application we run on this platform (MailManager) ... so any assistance you can give would be appreciated.:-) If I run fstest.py I get: # python /usr/local/Zope-2.7.0/utilities/ZODBTools/fstest.py /home/dballing/Data.fs /home/dballing/Data.fs has inconsistent transaction length for undone transaction at 475866827 .. if I try fsrefs, I get even less pleasant output: # python /usr/local/Zope-2.7.0/utilities/ZODBTools/fsrefs.py /home/dballing/Data.fs Traceback (most recent call last): File "/usr/local/Zope-2.7.0/utilities/ZODBTools/fsrefs.py", line 25, in ? from ZODB.FileStorage import FileStorage ImportError: No module named ZODB.FileStorage Ugh... any pointers on what I should do next? D _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Derek J. Balling wrote:
I'm resending this because a couple weeks later and not a single peep was to be heard, and my boss is starting to hound me for insight. :-(
Anyone have any helpful advice?
I would try on the zodb-dev list, where Tim Peters hangs out; he has more zen about packing problems than anybody else I know. Tres. -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com
[Derek J. Balling]
I'm resending this because a couple weeks later and not a single peep was to be heard, and my boss is starting to hound me for insight. :-(
Anyone have any helpful advice?
If your boss is willing to pay someone to help, that opens other possibilities. Otherwise this has to change:
K, now I don't mean to sound stupid, but I inherited a Zope install, and I know absolutely nothing about: (a) python (b) Zope (c) the application we run on this platform (MailManager)
That is, you'll have to teach yourself. Unfortunately, you're jumping in at the intersection of several deep ends: you have a corrupted Data.fs file, you don't have backups, and the problem first occurred a long time ago. We know that with high probability because:
If I run fstest.py I get:
# python /usr/local/Zope-2.7.0/utilities/ZODBTools/fstest.py /home/dballing/Data.fs /home/dballing/Data.fs has inconsistent transaction length for undone transaction at 475866827
If you read: http://zope.org/Wikis/ZODB/FileStorageBackup then you already know that .fs files grow only by appending, and your file is corrupt starting near byte 475,866,827. You originally said:
... Since the database is up to 5.X GB ...
so the damage occurred 4 or 5 gigabytes' worth of transactions in the past, or some process sprayed bytes to random spots on your disk (e.g., flaky disk controllers can exhibit that symptom, esp. under high load). When fstest.py fails, there's no (well, little) point to running fsrefs.py. It's possible you can recover by: 1. Bring your app down. 2. Make a physical copy of your Data.fs. 3. Learn how to use fsrecover.py, and try it on the copy. fstest.py should pass after that. fsrefs.py may or may not. There's no guarantee it will work: fsrecover.py proceeds simply by throwing away corrupted pieces of the FileStorage -- it's a very low-level program, with no semantic knowledge, so can create a well-formed file at a low level that's nevertheless missing objects crucial to your application. There's simply no magical fix for file corruption, whether it's a FileStorage or your email folder. It's possible (as Tres suggested) that someone reading the zodb-dev list would be able to volunteer more significant help than you're getting here.
I don't know if this suggestion is relevant, workable or advisable but I once got out of a hole by exporting a site (i.e. folder) to a zexp file. That was to do with a locked Version rather than a corrupt data.fs, and when the site was still quite small. You could try that by importing the zexp into a clean data.fs (if the site appears to work despite the corruption). Cliff Tim Peters wrote:
[Derek J. Balling]
I'm resending this because a couple weeks later and not a single peep was to be heard, and my boss is starting to hound me for insight. :-(
Anyone have any helpful advice?
If your boss is willing to pay someone to help, that opens other possibilities. Otherwise this has to change:
K, now I don't mean to sound stupid, but I inherited a Zope install, and I know absolutely nothing about: (a) python (b) Zope (c) the application we run on this platform (MailManager)
That is, you'll have to teach yourself. Unfortunately, you're jumping in at the intersection of several deep ends: you have a corrupted Data.fs file, you don't have backups, and the problem first occurred a long time ago. We know that with high probability because:
If I run fstest.py I get:
# python /usr/local/Zope-2.7.0/utilities/ZODBTools/fstest.py /home/dballing/Data.fs /home/dballing/Data.fs has inconsistent transaction length for undone transaction at 475866827
If you read:
http://zope.org/Wikis/ZODB/FileStorageBackup
then you already know that .fs files grow only by appending, and your file is corrupt starting near byte 475,866,827. You originally said:
... Since the database is up to 5.X GB ...
so the damage occurred 4 or 5 gigabytes' worth of transactions in the past, or some process sprayed bytes to random spots on your disk (e.g., flaky disk controllers can exhibit that symptom, esp. under high load).
When fstest.py fails, there's no (well, little) point to running fsrefs.py.
It's possible you can recover by:
1. Bring your app down. 2. Make a physical copy of your Data.fs. 3. Learn how to use fsrecover.py, and try it on the copy.
fstest.py should pass after that. fsrefs.py may or may not. There's no guarantee it will work: fsrecover.py proceeds simply by throwing away corrupted pieces of the FileStorage -- it's a very low-level program, with no semantic knowledge, so can create a well-formed file at a low level that's nevertheless missing objects crucial to your application.
There's simply no magical fix for file corruption, whether it's a FileStorage or your email folder.
It's possible (as Tres suggested) that someone reading the zodb-dev list would be able to volunteer more significant help than you're getting here. _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Tim Peters wrote:
[Derek J. Balling]
I'm resending this because a couple weeks later and not a single peep was to be heard, and my boss is starting to hound me for insight. :-(
Anyone have any helpful advice?
If your boss is willing to pay someone to help, that opens other possibilities. Otherwise this has to change:
My boss has authorized the use of cash to solve this problem. Scope would probably include "cleaning up the mess of the existing zope install, fixing the Data.fs file, and probably migrating to DirectoryStorage afterwards". If you're interested, please contact me off-list. D
Derek J. Balling wrote at 2004-12-1 16:47 -0500:
I'm resending this because a couple weeks later and not a single peep was to be heard, and my boss is starting to hound me for insight. :-(
Did you read "http://zope.org/Wikis/ZODB/FileStorageBackup"?
... If I run fstest.py I get:
# python /usr/local/Zope-2.7.0/utilities/ZODBTools/fstest.py /home/dballing/Data.fs /home/dballing/Data.fs has inconsistent transaction length for undone transaction at 475866827
Thus, your "Data.fs" is physically damaged. Copy it to a safe place. Then, you could try "fsrecover" to get part of your "Data.fs" back (about 475 MB). You probably will loose some data. Alternatives: * You may have a backup copy of "Data.fs" which is not yet damaged. * Packing leaves such a copy around as "Data.fs.old". * It is possible to get lots of data out of even a seriously damaged "Data.fs" but this requires a lot of work and expert knowledge.
.. if I try fsrefs, I get even less pleasant output: ... ImportError: No module named ZODB.FileStorage
Your PYTHONPATH does not include "<ZopeDir>/lib/python"... But, it is not worth to run "fsrefs" when "fstest" reports damage: "fstest" checks the physical structure, "fsrefs" the logical one. If the physical structure is damaged, the logical is damaged, too, with a very high probability. -- Dieter
Just a gloss on part of Dieter's excellent advice:
Then, you could try "fsrecover" to get part of your "Data.fs" back (about 475 MB).
Derek's .fs file is over 5GB. People tend to believe that fsrecover.py truncates a damaged .fs, but that's just because most people run it immediately after a power outage, or other circumstance in which just the tail end of the .fs is damaged. Then fsrecover does truncate it. But fsrecover *tries* to be smarter than that: it looks ahead after each damaged area it finds, for the first place it can "synch up" again. So it's possible that fsrecover will recover nearly the entire 5GB here; it's also possible that it will only recover 475MB.
You probably will loose some data.
That's certain, since fsrecover works by throwing bytes away. What's impossible to guess is whether it will throw away anything the app still needs. If the app is running is OK, the odds of success are decent.
participants (5)
-
Cliff Ford -
Derek J. Balling -
Dieter Maurer -
Tim Peters -
Tres Seaver