[Zope-dev] connection commit ordering

[Laurence Rowe]

On 18 June 2010 14:32, Leonardo Rochael Almeida <leorochael at gmail.com> wrote:
> Hi Laurence
>
> On Fri, Jun 18, 2010 at 08:06, Laurence Rowe <l at lrowe.co.uk> wrote:
>> On 18 June 2010 01:24, Leonardo Rochael Almeida <leorochael at gmail.com> wrote:
>>> By the way, this issue is completely separate from the
>>> two-phase-commit discussion that we had recently, since all the
>>> connectors involved here are fully transactional.
>>
>> As you can see here:
>>
>> http://zope3.pov.lt/trac/browser/Zope/trunk/src/Shared/DC/ZRDB/TM.py
>>
>>    def tpc_vote(self, *ignored):
>>        self._finalize = 1
>>
>>    def tpc_finish(self, *ignored):
>>
>>        if self._finalize:
>>            try: self._finish()
>>            finally: self._registered=0
>>
>> The transaction manager is only doing one phase commit. It sorts first
>> as it commits in the second phase. If you change the sort order, you
>> lose the guarantee of transactional integrity.
>
> For me this means that TM subclasses need to override tpc_vote and
> implement a proper commit preparation [1] [2] to assure they are
> correctly participating in the TPC dance.

zope.sqlalchemy does this, but that brings a whole orm into the
equation and does away with ZRDB legacy.

> And if that is not the case, but you have, for example, more than one
> MySQL connector, you are already in a situation where you can't
> guarantee transactional integrity, so this discussion is actually
> orthogonal to the sortOrder one.

That's true, but don't you see this problem even with only a single
ZODB and a single ZRDB connection?

>> Perhaps a better way to solve this would be to include the zope
>> transaction id in the table, then in the background thread only
>> reindex the queued items with a tid <= the current tid of the
>> connection.
>
> Possibly, but is there a way to know the id of a transaction that
> hasn't been committed yet, to store it on MySQL? Besides, when working
> with multiple mount points, you might have to store multiple TIDs, for
> all storages involved, or else there should be a global transaction ID
> that should be recorded everywhere, and I don't see the 'transaction'
> package providing one.

The ZODB storage's transaction id is set in tpc_begin, so you should
be able to get it in tcp_vote or tpc_finish of the ZRDB data manager.
Though doing so probably horribly complicates the ZSQLCatalog code.

> In any case, does anyone oppose the existence of a .setSortKey() on
> the TM class?

I don't oppose it, but I also don't see how this will fix the problem
unless you set the sort key to be greater than the ZODB's sort key.
This strikes me as a very bad idea for a TM that is designed to
tpc_finish before anything else.

Laurence