[ZODB-Dev] zodbupdate branch sylvain-persistent-load

[Christian Theune]

Hi,

On 02/01/2010 05:47 PM, Sylvain Viollon wrote:
> Hello,
> 
>    I did few month ago a branch of zodbupdate, which works completely 
> differently than the trunk. Instead of changing the pickle code of the record, 
> it unpickle it, changing references at that point, and repickle if there is 
> changes.
> 
>   I did this in order to support the ZODB references that are implemented 
> using persistent ID, in which a reference to a class can be expressed as a 
> tuple (module, classname) instead of the class pickle opcode. That tuple 
> (module, classname) can appear in the class meta pickle of a record as well. 
> You can have more details about this by reading the comments of the 
> serialize.py file in the ZODB code.
> 
>   If I understand correctly, those are legacy formats, that the ZODB doesn't 
> use anymore to write data, but is still able to read data stored using those 
> formats for backward compatibility support.
> 
>   The issue is that I want to use zodbupdate on database that have been 
> created a long time ago, and I do need zodbupdate to be able to read those 
> formats and correct class references in those records as well. That's the 
> motivation of my changes.
> 
>   My version of zodbupdate is feature alike with the old one: 
> 
> - it is able to update records which have backward compatibility imports in 
> the code, and report them at the end, even if they are not described in the 
> renaming rules,
> 
> -  it is able to ignore missing modules and classes (it use ZODB.broken),
> 
> - command line invocation stays the same,
> 
> - performance-wise, it seems even to be faster (I use cPickle),
> 
>   As well, it's able to work on 'broken data.fs' which trigger POSKey errors.
> 
>   I did my testing using the infrae.com website database, which have been 
> created around 2002 I think, and contain a lot of interesting cases.
> 
>   So I would like to release my work now, since I need to use it in production 
> environment. Since there is lot of changes, I can release it as a fork as 
> well, if it's not possible to merge it with the trunk.

I think I'm fine with your approach in general by now. I don't have the
time to review anything, though. The old version is happily there in SVN.

IIRC you hadn't updated all unit tests last time, did you catch up with
that yet?

Otherwise, feel free to merge and release.

Christian

-- 
Christian Theune · ct at gocept.com
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1
Zope and Plone consulting and development