[Zope] More on understanding conflicts

[Dennis Allison]

Zope 2.8.4
ZODB 3.4.2


I have a dtml method which is an obvious hotspot.  It puts up navigation
wigetry and is rendered on nearly every page.  The logs show it generates
a large number of conflicts.  The conflicts are in TemporaryStrorage and
so are due to session variables.

The session variables are accessed with short Script(Python) functions
one call per session variable used: getSessionVariable( var ),  Session 
variable hold mostly data that is constant throughout the session.  There 
are about twenty values, ten of which are referenced in the navigation 
method.

The structure of the method is simple enough: there is a large <dtml-let>
block which populates local variables with data from the session 
variables   

	<dtml-let foobar="getSessionVariable('foobar')"
         ...
         >
	
with the body of the <dtml-let> containing 300 lines or so of dtml, 
control code, sql queries, and occasional dtml-let blocks to hold 
query results and computed values.

So, this navigation method is clearly a good candidate for refactoring,
but the question is "What's the best way to refactor it?".

I still don't have an intuitve understanding of the conflict mechanism in
practice.  For example, Why is the conflict error reported for the
navigation dtml method rather than for the getSessionVariable python
script which actually references the SESSION object where the conflict 
occurs?
 
A conflict error gets raised whenever another thread references the same
session's SESSION object.  Apparently the MVCC read-read conflict
resolution fails for session variables although I am not sure why.  The
DEBUG logs indicate that read-read conflicts found.  It would be helpful
to understand why.

When the conflict is resolved, one is backed out (everything is 
transactional) and restarted.  After three retries, it is considered 
an error.   This processing is handled automatically by Zope.

At one point, I had considered writing a custom conflict resolution
function where resolution assumed independence of individual session
variables.  After some thought and experimentation, I concluded that was a
bad idea because it could break synchronization assertions between session
variables (e.g., X is twice Y).

Absent a magical conflict resolution method, the best approach would seem
to be to minimize the potential for conflicts.  Intuition says that the
simplest way to do this would be to have the navigation dtml method make a
single call to a python script, getSessionVariables, which would return a
copy of the session variables. The copy could then be unpacked and 
processed without any potential conflict errors.