[Zope-dev] Re: Conflict errors on BDBMinimal storage

[Chris McDonough]

On Tue, 2004-05-18 at 14:58, Chris McDonough wrote:
> On Tue, 2004-05-18 at 14:15, Tres Seaver wrote:
> > I recently reworked Anthony Baxter's SQLSessions product for a 
> > consulting customer, essentially modernizing the code and making it 
> > compliant with the new Sessions interface (I dropped the bits which do 
> > the work currently done by BrowserIDManager).  It took me about six 
> > hours or so, including writing unit tests.  Unfortunately, I don't have 
> > access to that code, due to the nature of the consulting arrangement.
> > 
> > What I would like to do is revisit the effort, using the new 
> > PluginRegistry code released as part of the PluggableAuthService 
> > release;  I am, however, pretty well booked up for the next few weeks. 
> > I would be glad to advise anybody who felt called to take on such an 
> > effort, however.
> 
> I would also like to see this happen.  I don't have the time to help
> implement this right now, but I may be able to work it in to an ongoing
> engagement at some point in the next few months.

In my testing, I'm able to see that naive sessioning applications (ones
that hit the session on every request, whether they really need to or
not) seem to fail down due to the "abort after 3 conflicts in a row"
hardcoded policy under extremely high load (~ 40-60 req/sec or so). 
This will happen regardless of what code I put in the sessioning
machinery; attempting to prevent it there is a lost cause: it gets a bit
like trying to prove the Heisenberg Uncertainty Principal or trying to
predict the future.

As a result, I am coming to believe that along with the "errors as part
of main" transaction patch for 2.7.1, I should also make the
retry-on-conflict-error policy pluggable for those who really
desperately need to slow their Zope systems to a conflict-induced
crawl.  They can bump it up to 100 as far as I care and if it works for
them, great.  The more selective people can wait around for a different
(non-ZODB-based) sessioning implementation or change their code to not
pound the snot out of the sessioning machinery unnecessarily.

As far as I'm concerned, the new Transience stuff is looking really good
for the common case.  I haven't been able to reproduce any of the
corruption issues that happened with the older implementation.

- C