I remember that 2.6 had issues with packing, but I was under the impression that they'd been fixed... However, attempting to pack a Data.fs in 2.6-current-cvs fails for me: File "/export/01/zope/dev_csr_code/lib/python/ZODB/DB.py", line 526, in pack try: self._storage.pack(t,referencesf) File "/export/01/zope/dev_csr_code/lib/python/ZODB/FileStorage.py", line 1503, in pack opos = p.pack() File "/export/01/zope/dev_csr_code/lib/python/ZODB/fspack.py", line 680, in pack self.gc.findReachable() File "/export/01/zope/dev_csr_code/lib/python/ZODB/fspack.py", line 464, in findReachable self.buildPackIndex() File "/export/01/zope/dev_csr_code/lib/python/ZODB/fspack.py", line 486, in buildPackIndex self.checkData(th, tpos, dh, pos) File "/export/01/zope/dev_csr_code/lib/python/ZODB/fspack.py", line 189, in checkData ": %d != %d", dh.tloc, tpos) File "/export/01/zope/dev_csr_code/lib/python/ZODB/fspack.py", line 170, in fail raise CorruptedError(s) ZODB.fspack.CorruptedError: /app/zope/dev_csr_server/var/Data.fs.packtest:9200838:data record does not point to transaction header: 17391715 != 9200760 This is reproducible. The particular Zope that uses this file was shut down cleanly before attempting to pack the DB. The code that fails to pack is: import ZODB from ZODB import FileStorage filename='/app/zope/dev_csr_server/var/Data.fs.packtest' store = FileStorage.FileStorage(filename) db = ZODB.DB(store) db.pack() # boom. Help? I'm using the ZODB that's in the current 2.6 branch of CVS - according to ZODB/__init__.py it's version 3.1.2. Is a newer version of ZODB going to help here? Anthony
Anthony Baxter wrote at 2003-8-12 18:18 +1000:
I remember that 2.6 had issues with packing, but I was under the impression that they'd been fixed...
However, attempting to pack a Data.fs in 2.6-current-cvs fails for me: ... raise CorruptedError(s) ZODB.fspack.CorruptedError: /app/zope/dev_csr_server/var/Data.fs.packtest:9200838:data record does not point to transaction header: 17391715 != 9200760
This is reproducible. The particular Zope that uses this file was shut down cleanly before attempting to pack the DB. The code that fails to pack is:
Maybe, the "Data.fs" was already corrupted before you switched to 2.6.2. Does the problem remain after a "fsrecover"? Dieter
I don't recall that any of the pack bugs that we fixed caused this sort of error. My first guess would be that the file is a bit damaged, perhaps in a way that the old pack did not check. Can you run fsdump.py and see if that output sheds any light. It will give you a detailed text summary of all the transactions and data records. You could use the dump to figure out if, for example, the two numbers in the error message are indeed transaction header locations. Jeremy
-----Original Message----- From: Anthony Baxter [mailto:anthony@interlink.com.au] Sent: Tuesday, August 12, 2003 4:18 AM To: zope-dev@zope.org Subject: [Zope-dev] Packing (still) hosed in 2.6.2?
I remember that 2.6 had issues with packing, but I was under the impression that they'd been fixed...
However, attempting to pack a Data.fs in 2.6-current-cvs fails for me:
File "/export/01/zope/dev_csr_code/lib/python/ZODB/DB.py", line 526, in pack try: self._storage.pack(t,referencesf) File "/export/01/zope/dev_csr_code/lib/python/ZODB/FileStorage.py", line 1503, in pack opos = p.pack() File "/export/01/zope/dev_csr_code/lib/python/ZODB/fspack.py", line 680, in pack self.gc.findReachable() File "/export/01/zope/dev_csr_code/lib/python/ZODB/fspack.py", line 464, in findReachable self.buildPackIndex() File "/export/01/zope/dev_csr_code/lib/python/ZODB/fspack.py", line 486, in buildPackIndex self.checkData(th, tpos, dh, pos) File "/export/01/zope/dev_csr_code/lib/python/ZODB/fspack.py", line 189, in checkData ": %d != %d", dh.tloc, tpos) File "/export/01/zope/dev_csr_code/lib/python/ZODB/fspack.py", line 170, in fail raise CorruptedError(s) ZODB.fspack.CorruptedError: /app/zope/dev_csr_server/var/Data.fs.packtest:9200838:data record does not point to transaction header: 17391715 != 9200760
This is reproducible. The particular Zope that uses this file was shut down cleanly before attempting to pack the DB. The code that fails to pack is:
import ZODB from ZODB import FileStorage filename='/app/zope/dev_csr_server/var/Data.fs.packtest' store = FileStorage.FileStorage(filename) db = ZODB.DB(store) db.pack() # boom.
Help? I'm using the ZODB that's in the current 2.6 branch of CVS - according to ZODB/__init__.py it's version 3.1.2. Is a newer version of ZODB going to help here?
Anthony
"Jeremy Hylton" wrote I don't recall that any of the pack bugs that we fixed caused this sort of error. My first guess would be that the file is a bit damaged, perhaps in a way that the old pack did not check. Can you run fsdump.py and see if that output sheds any light. It will give you a detailed text summary of all the transactions and data records. You could use the dump to figure out if, for example, the two numbers in the error message are indeed transaction header locations.
Ok. The original error, reformatted for clarity: ZODB.fspack.CorruptedError: /app/zope/dev_csr_server/recover/Data.fs:9200838: data record does not point to transaction header: 17391715 != 9200760 Here's the 3 transactions (one before, dead one, one after). There's definately a screwed transaction there. ============================================================ offset: 9198385 end pos: 9200752 transaction id: 033bb0f41f05f3aa trec len: 2367 status: ' ' user: ' jasonb' description: '/admin_tools/accounts/update_account/manage_edit' len(extra): 0 ------------------------------------------------------------ offset: 9198463 oid: 00000000000206bb revid: 033bb0f41f05f3aa previous record offset: 9193826 transaction offset: 9198385 len(data): 2247 redundant trec len: 2367 ============================================================ offset: 9200760 end pos: 9202513 transaction id: 033bb0f4b80ad42a trec len: 1753 status: 'u' user: ' jasonb' description: '/admin_tools/accounts/update_account/manage_edit' len(extra): 0 ------------------------------------------------------------ offset: 9200838 oid: 00000000000206bb revid: 033bb0f4b80ad42a previous record offset: 17389418 # XXXX woopsie transaction offset: 17391715 # XXXX woopsie len(data): 1633 redundant trec len: 1753 ============================================================ offset: 9202521 end pos: 9204883 transaction id: 033bb0f7be87736e trec len: 2362 status: ' ' user: ' jasonb' description: '/admin_tools/accounts/update_account/manage_edit' len(extra): 0 ------------------------------------------------------------ offset: 9202599 oid: 00000000000206bb revid: 033bb0f7be87736e previous record offset: 9198463 transaction offset: 9202521 len(data): 2242 redundant trec len: 2362 ============================================================ So, fixing this. fsrecover.py doesn't do the right thing, unfortunately. It just leaves the broken record alone. I'm not sure if making fsrecover handle this class of error is useful... It should be simply a matter of walking through the transactions, and patching the one that's busted. Or alternately, simply skipping it. I'll look into fixing this now. More in a bit. Anthony
participants (3)
-
Anthony Baxter -
Dieter Maurer -
Jeremy Hylton