[Zope-dev] Bulletproof ZCatalog proposal

[Phillip J. Eby]

At 02:07 PM 6/7/01 -0400, Shane Hathaway wrote:
>On Thursday 07 June 2001 12:17, Phillip J. Eby wrote:
>
>> The
>> only catch was that this would still produce conflicts at the head
>> end of the linked list.  :(  Of course, that was in the days before
>> ZODB conflict resolution.  Nowadays, you could probably implement it
>> as a simple sequence object with the conflict resolution method
>> implemented.  But then there'd be the question of how to resolve
>> conflicts if more than one thread or ZEO client decided to apply the
>> queued changes...  a 100 conflicts vs 1 situation.  Ugh.
>
>I was thinking the queue would only last the duration of a transaction 
>and that the queue would be thread-specific.

That's not hard to do - you could just toss an object into the transaction
queue to do it during transaction commit, similar to what ZPatterns does in
a more general way.  (Note that ZPatterns users can already get the
benefits of deferring catalog changes until the end of a (sub)transaction,
by using triggers to implement automatic cataloging).


>> Anyway, I'd be really interested in seeing "a solution for all the
>> issues in conflict resolution".
>
>I was thinking that certain types of objects would be committed by the 
>transaction manager before all others.  In this case, the catalog (or a 
>special object in the catalog) would be committed first.  It would 
>resolve all conflicts in the contained indices before they occur by 
>replaying the changes in the persisted queues from the transaction 
>history, then setting the _p_serial attributes to convince the storage 
>that the conflicts have already been resolved.


Hm.  Sounds to me like what you actually want is for the transaction
manager to do this *after* everything else, rather than before.  Thus, you
would catch any changes which occur *during* transaction commit - such as
commit-phase cataloging (as some folks do with ZPatterns currently).

That is, in ZPatterns one can specify triggers such as:

WHEN OBJECT DELETED, CHANGED CALL
someCatalog.manage_uncatalog(self.absolute_url(1))
WHEN OBJECT ADDED, CHANGED CALL
someCatalog.manage_catalog(self,self.absolute_url(1))

Which will be executed during transaction commit if the conditions apply.
If you have the catalog process its queue *before* other objects commit, it
will not have received these calls yet.  This will end up making the
catalog try to commit itself a second time, which will fail with a conflict
error - 100% of the time.  (And retry won't help, because the transaction
is conflicting with itself.)

Of course, this issue could be fixed by minor adjustment to the ZODB API
and implementation such that it is permissible to store() an object more
than once during a transaction's commit phase without creating a
ConflictError.