[ZODB-Dev] Setting the jar

[Tim Peters]

[Florent Guillaume]
>>> What part of the transaction/persistence code sets the _p_jar of a
>>> persistent object to that of its parent when the object is assigned to
>>> a persistent parent and transaction.commit(1) is called ?

[Tim]
>> I'm massively overloaded today, so it would really help if you could
>> name the version of ZODB you're wondering about ;-).

[Florent]
> No problem, there's no hurry.
>
> Latest ZODB 3.4 would do (I'm maintly interested in Zope 2.8 here.
>
> I've seen something that look like what I want in
> ZODB.Connection.Connection.add, but I can't see who calls it.

add() is a relatively new API, intended to be called "by hand" when someone
wants to force a new object to get an oid.  I'm not really clear about the
use case(s) that drove its addition, but it's not normally involved in
auto-assigning oids and/or jars.

> To be clear, I'm interested in how the _p_jar happens on a new object
> that's just been created. Basically if you do:
>
> # folder is a persistent object that has a _p_jar
> # folder.foo = Foo() # where Foo inherits from Persistent
> # here folder.foo doesn't have a _p_jar
> transaction.commit(1)
> # here folder.foo has a _p_jar, how come ?
>
> I think I've missed something in my code trawling ;)

Indeed, this will have you crawling on your belly for the next week <wink>.
In effect, it all happens as a side effect of pickling, via the
persistent_id() method of (in ZODB 3.4) class ZODB.serialize.ObjectWriter.
The class docstring is telling the truth:

    """Serializes objects for storage in the database.

    The ObjectWriter creates object pickles in the ZODB format.  It
    also detects new persistent objects reachable from the current
    object.
    """

but exactly how that happens requires tracing thru Python's C pickle code
too.  Stick a print or a breakpoint here (inside persistent_id()) and you
can watch it happening:

        if oid is None:
            oid = obj._p_oid = self._jar.new_oid()
            obj._p_jar = self._jar
            self._stack.append(obj)

Note that ObjectWriter is passed a single object, but
Connection._store_objects iterates over the ObjectWriter instance.  In your
example, "folder.foo = Foo()" marks `folder` as changed, and the new object
bound to .foo is discovered while pickling the new state for `folder`.
Iterating over an ObjectWriter(obj) instance can end up discovering an
arbitrarily large and complex graph of new objects reachable from obj, as it
tries to pickle each in turn.  That's what the `self._stack` in the above is
keeping track of.