On 18 June 2010 14:32, Leonardo Rochael Almeida <leorochael@gmail.com> wrote:
Hi Laurence
On Fri, Jun 18, 2010 at 08:06, Laurence Rowe <l@lrowe.co.uk> wrote:
On 18 June 2010 01:24, Leonardo Rochael Almeida <leorochael@gmail.com> wrote:
By the way, this issue is completely separate from the two-phase-commit discussion that we had recently, since all the connectors involved here are fully transactional.
As you can see here:
http://zope3.pov.lt/trac/browser/Zope/trunk/src/Shared/DC/ZRDB/TM.py
def tpc_vote(self, *ignored): self._finalize = 1
def tpc_finish(self, *ignored):
if self._finalize: try: self._finish() finally: self._registered=0
The transaction manager is only doing one phase commit. It sorts first as it commits in the second phase. If you change the sort order, you lose the guarantee of transactional integrity.
For me this means that TM subclasses need to override tpc_vote and implement a proper commit preparation [1] [2] to assure they are correctly participating in the TPC dance.
zope.sqlalchemy does this, but that brings a whole orm into the equation and does away with ZRDB legacy.
And if that is not the case, but you have, for example, more than one MySQL connector, you are already in a situation where you can't guarantee transactional integrity, so this discussion is actually orthogonal to the sortOrder one.
That's true, but don't you see this problem even with only a single ZODB and a single ZRDB connection?
Perhaps a better way to solve this would be to include the zope transaction id in the table, then in the background thread only reindex the queued items with a tid <= the current tid of the connection.
Possibly, but is there a way to know the id of a transaction that hasn't been committed yet, to store it on MySQL? Besides, when working with multiple mount points, you might have to store multiple TIDs, for all storages involved, or else there should be a global transaction ID that should be recorded everywhere, and I don't see the 'transaction' package providing one.
The ZODB storage's transaction id is set in tpc_begin, so you should be able to get it in tcp_vote or tpc_finish of the ZRDB data manager. Though doing so probably horribly complicates the ZSQLCatalog code.
In any case, does anyone oppose the existence of a .setSortKey() on the TM class?
I don't oppose it, but I also don't see how this will fix the problem unless you set the sort key to be greater than the ZODB's sort key. This strikes me as a very bad idea for a TM that is designed to tpc_finish before anything else. Laurence