[ZODB-Dev] zodbupdate branch sylvain-persistent-load
Sylvain Viollon
sylvain at infrae.com
Mon Feb 1 11:47:15 EST 2010
Hello,
I did few month ago a branch of zodbupdate, which works completely
differently than the trunk. Instead of changing the pickle code of the record,
it unpickle it, changing references at that point, and repickle if there is
changes.
I did this in order to support the ZODB references that are implemented
using persistent ID, in which a reference to a class can be expressed as a
tuple (module, classname) instead of the class pickle opcode. That tuple
(module, classname) can appear in the class meta pickle of a record as well.
You can have more details about this by reading the comments of the
serialize.py file in the ZODB code.
If I understand correctly, those are legacy formats, that the ZODB doesn't
use anymore to write data, but is still able to read data stored using those
formats for backward compatibility support.
The issue is that I want to use zodbupdate on database that have been
created a long time ago, and I do need zodbupdate to be able to read those
formats and correct class references in those records as well. That's the
motivation of my changes.
My version of zodbupdate is feature alike with the old one:
- it is able to update records which have backward compatibility imports in
the code, and report them at the end, even if they are not described in the
renaming rules,
- it is able to ignore missing modules and classes (it use ZODB.broken),
- command line invocation stays the same,
- performance-wise, it seems even to be faster (I use cPickle),
As well, it's able to work on 'broken data.fs' which trigger POSKey errors.
I did my testing using the infrae.com website database, which have been
created around 2002 I think, and contain a lot of interesting cases.
So I would like to release my work now, since I need to use it in production
environment. Since there is lot of changes, I can release it as a fork as
well, if it's not possible to merge it with the trunk.
Regards,
Sylvain,
--
Sylvain Viollon -- Infrae
t +31 10 243 7051 -- http://infrae.com
Hoevestraat 10 3033GC Rotterdam -- The Netherlands
More information about the ZODB-Dev
mailing list