[Zope-dev] Re: sessions in the presence of conflicts

[Chris McDonough]

Trimmed zodb-dev off the cc list.

On Dec 15, 2005, at 2:24 PM, Dennis Allison wrote:

> The systems are running a Zope/ZEO combination with a store  
> configuration
> of:
>
> #
>  <zodb_db temporary>
>      # Temporary storage database (for sessions)
>      <temporarystorage>
>        name temporary storage for sessioning
>      </temporarystorage>
>      mount-point /temp_folder
>      container-class Products.TemporaryFolder.TemporaryContainer
>  </zodb_db>

OK, that's good.  Having a nonzeo session database is a good thing.
>

> Although the connection to ZEO is via a network port, it runs on  
> the same
> physical hardware.
>
> TemporaryStorage is not transactional.  Does it need to be under MVCC?

I thought we sewed this one up. ;-)  It is indeed transactional.  See  
our email conversations around 11/15/2005 and beyond.

>
> TemporaryStorage does provide a conflict cache to do "rudimentary  
> conflict
> resolution".  There are several timing and scaling parameters that  
> need to
> be considered:
>
> CONFLICT_CACHE_MAXAGE = 60  (seconds)
> CONFLICT_CACHE_GCEVERY = 60 (seconds)
> RECENTLY_GC_OIDS_LEN = 200
>
> Entries in the recently gc's oids list are those which may be  
> resolvable
> by a retry.
>
> These numbers may be too small given the loads we see and the  
> number of
> accesses made to the session variables.  I plan to incrase them to see
> if there is any impact.

That's fine.
>
> MAYBE CONFLICTS AND THEIR RESOLUTION ARE NOT THE ROOT CAUSE OF THE  
> SESSION
> VARIABLE PROBLEM.

That's possible.


> either all the session variables
> are gone and only the container remains or most of the session  
> variables
> are gone and a few remain.

The former sounds like the session may have expired.  The second  
sounds like it could be due to conflict resolution.  You don't say  
where the KeyError is raised from, that's important.

>
> 74769573A2H7SIH2AKo=id: 11343269231636975299, token:  
> 74769573A2H7SIH2AKo,
> contents keys: ['currentTab', 'calendarPage', 'currentCourse',
> 'currentTextbook']
>
> and
>
> 77307574A2HTTdXCYYg=id: 11343267811075063138, token:  
> 77307574A2HTTdXCYYg,
> contents keys: []

Note that these are actually two different session objects (see the  
"token").  I don't know if this is meaningful.

>
> Access to the session variables are almost alwsys through a pair of
> Scripts(Python).  Occasionally a session variable is read with an
> expression of the form REQUEST['SESSION']['key'].
>
> ## Script (Python) "getSessionVariable"
> ##bind container=container
> ##bind context=context
> ##bind namespace=
> ##bind script=script
> ##bind subpath=traverse_subpath
> ##parameters=varname
> ##title=
> ##
> request=container.REQUEST
> session=request['SESSION']
> return session[varname]


How about session.get('varname') instead?  What happens when the  
session expires?  Do you deal with that in your code in some way?

>
> # Script (Python) "setSessionVariable"
> ##bind container=container
> ##bind context=context
> ##bind namespace=
> ##bind script=script
> ##bind subpath=traverse_subpath
> ##parameters=var, val
> ##title=
> ##
> request = container.REQUEST
> RESPONSE =  request.RESPONSE
> session=request['SESSION']
> session[var]=val
> request.set( 'SESSION', session )
>
> This all seems right to me.  Any suggestions as to how to localized  
> when
> the session variables get lost?  That might help localize the root  
> cause.
>

What is request.set('SESSION', session) supposed to do?  Whatever  
it's supposed to be doing is probably not what you think it's doing.   
I actually don't know quite what the impact of doing that would be;  
it would depend on your other code.  Remove that, it can't do  
anything good. ;-)  If you're trying to make sure the session is "re- 
stored" in the ZODB, you needn't.  You only need to do this for  
mutable variables *in* the session, eg.:

request.SESSION['a'] = {}
a = request.SESSION['a']
a['b'] = 'c'
# here it comes
request.SESSION['a'] = a

I also enumerated other knobs to you in previous emails that could  
help reduce sessioning conflicts (like turning off inband  
housekeeping -- see the Transience.py code for that -- and bumping up  
transient object container timeout resolution via session-timeout- 
resolution in zope.conf in your current config).  Those are the  
things I meant when I said that you hadn't told us whether you had  
taken steps based on recommendations.  I won't enumerate them here  
again here as it should be reasonably easy to go look in the archives  
or your mail history for those recommendations.  I'd suggest trying  
these recommendations before playing around with conflict resolution  
code.  Dunny also suggested trying to mess with the WRITEGRANULIARITY  
in TransientObject.py, FWIW.

- C