[ZODB-Dev] ZODB alternate serialization format patch
Hanno Schlichting
hannosch at hannosch.eu
Fri Jan 16 04:27:02 EST 2009
Shane Hathaway wrote:
> Hanno Schlichting wrote:
>> Shane Hathaway wrote:
>>> In addition to making the data format pluggable, the patch puts most of
>>> ZODB's dependencies on the cPickle module in one place, so now if we
>>> decide to improve our usage of the cPickle module, we can make that
>>> change in one place rather than several.
>> I'm wondering if this would be a good opportunity to include a version
>> marker in addition to the protocol format to the API's?
>
> Maybe. Starting with pickle protocol version 2, pickles do start with a
> version header.
Reading pickles is backwards compatible. But specifying the version in
which to write pickles needs explicit configuration.
>> The ZODB still uses version one of the pickle protocol throughout and so
>> far it has been complicated and cumbersome to change this in any way.
>
> Is it cumbersome? I tried changing to pickle protocol v2 while making
> this patch.
Without this patch it was cumbersome, as the number of places to change
the pickle version was many. Now it probably has become a lot easier ;)
> Only 4 of the 3000+ ZODB tests failed. One of them failed
> because we have some pickle introspection code that does not yet
> understand protocol 2; that should be easy to fix. Another failed
> because it seems to depend on the exact length of a generated pickle,
> which is a bit silly. I didn't look at the rest of the test failures
> because I assume they are all similarly superficial.
That sounds easy to fix.
>> While the documentation of Protocol Buffers mention that they try to be
>> stable and avoid incompatible versions, I don't trust any standard to be
>> so generic that it can avoid incompatible changes over a period of many
>> years.
>
> The patch I made might cover this. After this patch, ZODB will look for
> the serializer format name in curly braces at the beginning of each
> serialized object. That format name could include a version number if
> necessary.
Right. I was wondering if it would be a good idea to build this in from
the start. My impression is that every data format of any kind of
non-trivial complexity will have multiple incompatible versions of the
same spec at some point. Maybe it is YAGNI as the formats themselves
have a version mechanism in them, though.
Hanno
More information about the ZODB-Dev
mailing list