[ZODB-Dev] Re: Fixing Broken Objects?

[Chris Spencer]

Tim Peters wrote:
> [Chris Spencer]
> [...]
> 
>>I'm fully aware of Pickle's limitations. My question was why are you
>>accepting them?
> 
> 
> Regardless of topic, the answer to that question is usually "although I
> don't have time for it in any case, if I did I can think of 100 other things
> likely to have better payoff" <0.5 wink>.
> 
> 
>>What's to stop someone from extending Pickle to serialize state
>>*and* code?
> 
> 
> You can do that right now if you're willing to store source code as string
> data.  There's much less attraction to storing CPython byte code:
> 
> - CPython byte code changes across Python releases:  there's no
>   guarantee that a .pyc file created by an older release will run
>   under a later release.  In contrast, pickles have always been
>   100% transportable from older to newer releases.  Needing to add
>   a "oh, unless you pickled a code object" caveat wouldn't be loved.
> 
> - You're wishing away all the problems involving imports.  The
>   CPython byte code for, e.g.,
> 
>     from zope.app.i18n import ZopeMessageFactory as _
> 
>   contains the package paths and function names as plain strings.
>   How is that supposed to execute if the source code for _those_
>   doesn't exist?  There are ways, but they're difficult and
>   delicate.
> 
> - Very few people _want_ to run without textual source code.
>   Tracebacks are much less useful then.  Python debuggers are much
>   less useful then.  You can't look at the code to see exactly what
>   it does if you have a question.  You can't sanely patch the code
>   to worm around a bug.  You can't get on IRC and say "help me
>   understand what's going on around line 237".
> 
> - Almost all the tools Python users rely on become similarly
>   useless, from source control systems to Emacs.
> 
> - It's generally thought to be "a feature" that pickle does _not_
>   store class implementation code along with that class's instances.
>   That's why the Python docs say:
> 
>       This is done on purpose, so you can fix bugs in a class or
>       add methods to the class and still load objects that were
>       created with an earlier version of the class.
> 
>   That's very useful.  Of course it creates problems too.  So
>   would not being able to "fix bugs ... and still load objects
>   that were created with an earlier version ..." easily.
> 

You'll get no argument from me on these points. In fact, I'd strongly 
prefer storing the source code as a string, as opposed to a 
byte-compiled code object. The intent would be to still allow source 
code editing.

As Michel Pelletier pointed out, Pyro already supports a similar concept 
through what it calls "mobile code". Unfortunately, Pyro's oriented 
around remote-procedures, not persistence, so there's still a lot of 
work involved with getting it working on a level with ZODB.

>>Python's inspect module certainly allows for this.
> 
> 
> Sorry, I didn't understand that (the preceding sentence asked what's to stop
> someone from extending pickle, and that's not a reasonable referent for
> "this").
> 

I meant the inspect module could be used to retrieve code for storage on 
the database, which is what I would extend the concept of pickling to 
do. You could of course do this manually, by storing the code along with 
the object on the server. Then, if you get a broken object, you look for 
and load the source code, then reload the object. This is basically what 
Pyro does, and I'm sure it wouldn't be too difficult to automate a 
similar procedure in ZODB.

Chris