[ZODB-Dev] Towards ZODB on Python 3

[Jim Fulton]

On Sun, Mar 10, 2013 at 12:13 PM, Tres Seaver <tseaver at palladion.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 03/10/2013 11:55 AM, Jim Fulton wrote:
>> On Sun, Mar 10, 2013 at 11:25 AM, Tres Seaver <tseaver at palladion.com>
>> wrote:
>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>>>
>>> On 03/10/2013 09:19 AM, Jim Fulton wrote:
>> ...
>>>> I think the fix is pretty straightforward.
>>>>
>>>> In the default __setstate__ provided by Persistent, and when
>>>> loading non-persistent instances:
>>>>
>>>> - On Python 2, ASCII encode unicode attribute names.
>>>>
>>>> - On Python 3, ASCII decode byte attribute names.
>>>>
>>>> The same transformation is necessary when looking up global
>>>> names.
>>>
>>> Hmm, if zodbpickle has to handle the issue for non-persistent
>>> instances and global names, wouldn't it be simpler to make it handle
>>> persistent instances too?
>>
>> No.  It can't know when a key is going to be used for a persistent
>> attribute name.
>>
>>> It can examine the stack inside 'load_dict' to figure out that the
>>> context is an instance, right?
>>
>> Ugh.  What stack?
>
> The one where the unpickler keeps its work-in-progress?
>
>  static int
>  load_none(UnpicklerObject *self)
>  {
>      PDATA_APPEND(self->stack, Py_None, -1);
>      return 0;
>  }
>
>  static int
>  load_dict(UnpicklerObject *self)
>  {
>      PyObject *dict, *key, *value;
>      Py_ssize_t i, j, k;
>
>      if ((i = marker(self)) < 0)
>          return -1;
>      j = Py_SIZE(self->stack);
>
>      if ((dict = PyDict_New()) == NULL)
>          return -1;
>
>      for (k = i + 1; k < j; k += 2) {
>          key = self->stack->data[k - 1];
>          value = self->stack->data[k];
>          if (PyDict_SetItem(dict, key, value) < 0) {
>              Py_DECREF(dict);
>              return -1;
>          }
>      }
>      Pdata_clear(self->stack, i);
>      PDATA_PUSH(self->stack, dict, -1);
>      return 0;
>  }

That won't work for persistent objects.

Persistent state is set by the deserializer, not by the unpickler.

The deserializer calls the unpickler to load the state.  It then calls
__setstate__ on the persistent object to set the state.  The
serializer doesn't know how to interpret the state, only __setstate__
does.

Jim

--
Jim Fulton
http://www.linkedin.com/in/jimfulton
Jerky is better than bacon! http://zo.pe/Kqm