Corrupt Data.fs: Is there a way of fixing transaction ids?
Hi! I have a really large Data.fs file (1.3 GB) that has a number of errors. When I run the fstest.py script, I get this: Problem: 109963117 object serialno 0x034573c4b6a2cb6e does not match transaction id 0x034573c55c8c0dbb How can I fix the Data.fs file? It should be possible to scan the Data.fs and create correct serialnos or transaction ids for the broken entries. But how exactly would I do that? I'd also like to identify the corrupt entries (i.e. find out the Zope object they belong to) to be able to eliminate them. My ultimate question is how these corrupt entries can exist at all. The Data.fs hit the 2 GB border once. So that could be a reason. But even then it would be really nice for the ZODB to not write corrupt entries ... Any help is appreciated ... Thanks Joachim
Joachim Werner <joe@iuveno.de> wrote:
Problem: 109963117 object serialno 0x034573c4b6a2cb6e does not match transaction id 0x034573c55c8c0dbb [...] I'd also like to identify the corrupt entries (i.e. find out the Zope object they belong to) to be able to eliminate them.
Here's a guess (I don't know if it'll work): Convert the hexadecimal id to octal, and do: import Zope app = Zope.app() ob = app._p_jar['\000\000\000\000\000(\347\026'] # here the octal string print ob.id print ob.__class__ HTH, Florent -- Florent Guillaume, Nuxeo (Paris, France) +33 1 40 33 79 87 http://nuxeo.com mailto:fg@nuxeo.com
"JW" == Joachim Werner <joe@iuveno.de> writes:
JW> Hi! I have a really large Data.fs file (1.3 GB) that has a JW> number of errors. When I run the fstest.py script, I get this: JW> Problem: 109963117 object serialno 0x034573c4b6a2cb6e does not JW> match transaction id 0x034573c55c8c0dbb JW> How can I fix the Data.fs file? It should be possible to scan JW> the Data.fs and create correct serialnos or transaction ids for JW> the broken entries. But how exactly would I do that? JW> I'd also like to identify the corrupt entries (i.e. find out the JW> Zope object they belong to) to be able to eliminate them. I'd start by running fsdump.py and identifying what transaction and objects are affected by this problem. fsdump will display transaction metadata (like time) and oid & class for each object. JW> My ultimate question is how these corrupt entries can exist at JW> all. The Data.fs hit the 2 GB border once. So that could be a JW> reason. But even then it would be really nice for the ZODB to JW> not write corrupt entries ... I agree that ZODB should not write the corrupt entries. It's possible, of course, that it didn't. It's certainly possible for bit-rot to strike a Data.fs, although it seems odd that the only manifestation of a disk failure would be a mismatch between object serialno and transaction id. I might be able to help more by looking at the actual Data.fs. Are you in a position to share it? (I can be discrete ;-). Jeremy
Hi! Unfortunately fsrecover.py is a bit too "primitive" for that task. It will just handle problems at the very end of the ZODB and cut the ZODB off when it finds an error. That's exactly what I can't do because that would mean that I loose most of my data. The tool that seems to be missing for the ZODB is something that really fixes a particular broken entry in the ZODB, not just identifies it. Cheers Joachim ----- Original Message ----- From: "Dieter Maurer" <dieter@handshake.de> To: "Joachim Werner" <joe@iuveno.de> Cc: <zope-dev@zope.org> Sent: Sunday, August 04, 2002 11:42 PM Subject: Re: [Zope-dev] Corrupt Data.fs: Is there a way of fixing transaction ids?
Joachim Werner writes:
... How can I fix the Data.fs file? Maybe, you can use "fsrecover.py". It lives in the ZODB folder.
Dieter
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
well, fsrecover helped me in a case just like yours. it does more than just cut off Data.fs, it actually repairs records as well. it's not as primitive as you think. just ty it out. make a copy of your Data.fs and run fsrecover.py on it. for me it came back saying "0 bytes truncated" and at first i thought the file had not changed. but it had and fstest did not find problems anymore. jens On Monday, August 5, 2002, at 04:52 , Joachim Werner wrote:
Hi!
Unfortunately fsrecover.py is a bit too "primitive" for that task. It will just handle problems at the very end of the ZODB and cut the ZODB off when it finds an error. That's exactly what I can't do because that would mean that I loose most of my data.
The tool that seems to be missing for the ZODB is something that really fixes a particular broken entry in the ZODB, not just identifies it.
Cheers
Joachim
----- Original Message ----- From: "Dieter Maurer" <dieter@handshake.de> To: "Joachim Werner" <joe@iuveno.de> Cc: <zope-dev@zope.org> Sent: Sunday, August 04, 2002 11:42 PM Subject: Re: [Zope-dev] Corrupt Data.fs: Is there a way of fixing transaction ids?
Joachim Werner writes:
... How can I fix the Data.fs file? Maybe, you can use "fsrecover.py". It lives in the ZODB folder.
Dieter
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
participants (5)
-
Dieter Maurer -
Florent Guillaume -
Jens Vagelpohl -
jeremy＠zope.com -
Joachim Werner