[ZODB-Dev] sessions in the presence of conflicts

[Dennis Allison]

Zope 2.8.4, ZODB 3.4.2

I am still trying to resolve a "session variables in the presence of
conflicts" problem.  The observed symptom is that suddenly either some
session variables disappear or all session variables disappear.  In both
cases, Zope has raised a KeyError exception for the a particular session
variable because it is not present and an attempt was being made to access
it from the session.

The cause appears to be related to the presence of a conflict error and a
botched recovery, but the exact mechanism remains a mystery.

Zope equates a session object with a TransientObject.  For our instance,
that's fine, since our only TransientObject is a session.  For the general
case, I would have thought a SessionObject inheriting from TransiendObject
would have been used so that conflict resolution for sessions could be
easily specialized.

The conflict resolution for a TransientObject (that is, a Session Object)
is clearly suboptimal.  In the next paragraphs I'll review what is 
currently done and then propose a different approach which I am about to 
implement for our systems.  Comments, corrections,  and suggestions much 
appreciated.

_p_resolveConflict(self, oldState, savedState, newState) 

returns the state of the object after resolving different changes.  The 
arguments are:

   oldState   -- state of the object at the beginning of the current 
	         transaction (mutable)
   savedState -- state currently stored in the database.  This state 
                 was written after oldState and reflect changes made
                 by a transaction that committed before the current
                 transaction (immutable)
   newState   -- state after changes made by the current transaction 
	         (immutable)

The standard conflict resolution for a TransientObject resolves according
to the following rules:

   1.  if any of the states are invalid (that is, has a key '_invalid')
       return the invalid state.

   2.  if any any of the attributes ['token','id','_created'] differ then 
       there is a conflict, raise the conflict exception.

   3.  choose the state most recently modified, if possible.

   4.  otherwise, choose the state most recently accessed.

It seems to me that we can do much better for sessions because we know a
bit about the semantics of sessions.  A session object is a
dictionary-like object mapping key-value pairs.  Adding or deleting keys
or changing the value associated with a key are independent operations and
do not conflict unless the keys are duplicated in both the transactions.  
Any conflict resolution mechanism needs to be able to manage multiple keys
independently since the session object is modified as a unit.   In 
addition, new keys may be added and old keys deleted; any conflict 
resolution mechanism at the key level needs to be comprehend those 
operations.

A more session-friendly conflict resolution might use:

   1.  if any of the states are invalid (that is, has a key '_invalid')
       return the invalid state.

   2.  if any any of the states attributes ['token','id','_created']
       differ then there is a conflict, raise the conflict exception. 

   3.  order the newState and savedState by modification time (or if that
       cannot be computed, by access time).   

   4.  any key appearing in oldState's dictionary but not appearing in 
       both savedState and newState should be removed from all.  This 
       corresponds to a key-value pair being deleted in one of the 
       transactions.  Insertions will be managed automatically by 
       the updates.

   5.  beginning with the oldest, update oldState dictionary of key-value 
       pairs using the dictionary part of newState and savedState.  Return 
       oldState.

This does several things.  First, it captures independent key-value
changes made in both potentially conflicting transactions.  Second, it
provides a reasonable ordering for multiple (potentially conflicting)
key-value pair updates.  Third, it manages insertions and deletions to the
session variable set in the presence of conflicts.

Does this make sense?  I have yet to figure out how to map a
TransientObject "state" back to the object it represents, but it clearly
is possible.