[ZODB-Dev] [zopefoundation/ZODB] 49919d: test for POSKeyError during transaction commit
Jim Fulton
jim at zope.com
Wed Feb 5 15:44:39 CET 2014
On Wed, Feb 5, 2014 at 9:23 AM, Godefroid Chapelle <gotcha at bubblenet.be> wrote:
> Le 05/02/14 13:25, Jim Fulton a écrit :
>
>>>>> Acquisition is added as a test dependency. Any hint how to replicate
>>>>> >> >the bug without acquisition is welcome.
>>>>
>>>> >>
>>>> >>Define a subclass of Persistent which emulates what Acquisition does,
>>>> >> e.g.:
>>>> >>
>>>> >> from persistent import Persistent
>>>> >> class Foo(Persistent):
>>>> >> @property
>>>> >> def _p_jar(self): # or whatever attribute trggers
>>>> >> return object()
>>>
>>> >
>>> >What if full replication requires a C extension module?
>>> >
>>> >(I hope that's not true and that it is possible to reproduce the bug
>>> >using some fakes, but I haven't spent the time investigating this.)
>>
>> I'm going to dig into this. I'm baffled by the assertion that this has
>> anything to do with readCurrent.
>
>
> For sure : the POSKeyError happens during connection.commit when checking
> oids stored in Connection._readCurrent mapping.
>
> (see traceback at http://rpatterson.net/blog/poskeyerror-during-commit)
>
> The _readCurrent mapping is populated only by calls to
> Connection.readCurrent method.
>
> In the Plone code base, the only way I found to get that
> Connection.readCurrent method to be called is by adding a key value pair to
> a BTree.
>
> _BTree_set C function is then called, which in turn calls readCurrent by
> inlining the PER_READCURRENT macro.
>
> This calls the cPersistence.c readCurrent function, which in turn calls
> readCurrent method on the ZODB connection.
Wow. I had to dig a bit to remind myself (vaguely) why I added this.
>
> When setting a key value pair on a new (not already committed) instance of a
> standard BTree, readCurrent method is not called on the connection.
This is with your change, right?
>
> My understanding is that it is due to the fact that _p_jar and _p_oid are
> only set during transaction commit.
They can be set earlier by calling the connection add method.
This is used often for frameworks that use object ids at the application level.
>
> However, with a new BTree instance that also inherits from
> Acquisition.Implicit, readCurrent method is called on ZODB connection when
> setting key value pair. The only explanation I found is that this instance
> _p_jar attribute has a value (acquired in a way or another ?).
You could also simulate this by adding an object to a connection using
a connection's add method.
Wanna update the test to use this technique instead?
>
> In this case, when readCurrent is called on an object created during a
> savepoint and this savepoint is rolled back, the oid is leftover in the
> Connection._readCurrent mapping. This leads to the POSKeyError when
> committing later as checkCurrentSerialInTransaction cannot check the object
> since it went away at rollback.
>
> This brings us to the fix I propose: calls to readCurrent should not track
> objects with oid equal to z64.
...
> This was a very long explanation which I hope will help to confirm the fix
> or to come up with a better one.
>
> PS: keep in mind that english is not my mothertongue.
:) You do very well.
I think your fix is correct. As you point out, It doesn't make sense to
guard against conflicts on new objects.
I think a cleaner test could be written using the connection add method.
Jim
--
Jim Fulton
http://www.linkedin.com/in/jimfulton
More information about the ZODB-Dev
mailing list