[ZODB-Dev] Re: Invalid object references on brand new Data.fs using fsrefs.py

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tim Peters wrote:
> [various fsrefs.py failures in Zope 2.9.3]
> 
> [Tres]
>> I can reproduce this on the Zope 2.9 branch, but not on the 2.8 branch.
> 
> I'll note that it can be reproduced easily with a standalone ZODB 3.6 too:
> 
> """
> import ZODB
> import ZODB.FileStorage
> import transaction
> from BTrees.OOBTree import OOBTree
> 
> st = ZODB.FileStorage.FileStorage("Data.fs")
> db = ZODB.DB(st)
> cn = db.open()
> rt = cn.root()
> 
> rt["tree"] = OOBTree()
> transaction.commit()
> db.close()
> """
> 
> Run that, and then:
> 
> $ python24\python.exe src\scripts\fsrefs.py Data.fs
> oid 0x0L persistent.mapping.PersistentMapping
> last updated: 2006-07-27 01:37:35.342000, tid=0x36715E196CAE377L
> refers to invalid object:
>        oid ('\x00\x00\x00\x00\x00\x00\x00\x01', None) missing: '<unknown>'
> 
>> ...
>> My *guess* is that this represents a failure in the fsrefs script, since
>> the appserver is functioning properly.  Perhaps it has not been updated
>> to deal with the API changes between ZODB 3.4.x and ZODB 3.6.x?  The
>> delta between the two is pretty small::
> 
> I expect it was actually the /entire/ checkin that made that change to
> fsrefs.py:
> 
>    http://svn.zope.org/ZODB/trunk/src/ZODB/serialize.py?rev=30715&view=rev
> 
> That moved and changed the crucial get_refs() function used by
> fsrefs.py too.  Unfortunately, I don't believe there are any tests for
> fsrefs, and I bet it's been broken ever since that checkin.
> 
> This code in fsrefs.py doesn't make sense now:
> 
>            ref, klass = info
>            if klass is None:
>                # failed to unpack
>                ref = info
>                klass = '<unknown>'
> 
> The "failed to unpack" comment is left over from before that checkin,
> where it did make sense.  The "ref = info" after it appears to be dead
> wrong.  Try this patch?:
> 
> Index: src/scripts/fsrefs.py
> ===================================================================
> --- src/scripts/fsrefs.py       (revision 69267)
> +++ src/scripts/fsrefs.py       (working copy)
> @@ -129,11 +129,8 @@
>         data, serial = fs.load(oid, "")
>         refs = get_refs(data)
>         missing = [] # contains 3-tuples of oid, klass-metadata, reason
> -        for info in refs:
> -            ref, klass = info
> +        for ref, klass in refs:
>             if klass is None:
> -                # failed to unpack
> -                ref = info
>                 klass = '<unknown>'
>             if ref not in fs._index:
>                 missing.append((ref, klass, "missing"))
> _______________________________________________
> For more information about ZODB, see the ZODB Wiki:
> http://www.zope.org/Wikis/ZODB/
> 
> ZODB-Dev mailing list  -  ZODB-Dev at zope.org
> http://mail.zope.org/mailman/listinfo/zodb-dev

OK, I'll apply that patch on the 3.6 and 3.7 branches and the trunk, as
it resolves that issue AFAICT.


Tres.
- --
===================================================================
Tres Seaver          +1 202-558-7113          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFEySij+gerLs4ltQ4RAqQiAJ9qnnzM9aUo2LM2VGjjV+Xhe5FuygCeJzYp
iRwe/LnhKYpOl6YTNBcy9V0=
=4Cg1
-----END PGP SIGNATURE-----