[ZODB-Dev] Re: BTrees strangeness (was [Zope-dev] Zope 2.X BIG Session problems - blocker - our site dies - need help of experience Zope developer, please)

[Michael Dunstan]

On 11/05/2004, at 4:54 PM, Chris McDonough wrote:

> On Mon, 2004-04-26 at 18:34, Michael Dunstan wrote:
>> I removed all the above hacks leaving just the
>> one that comments out the construction of the to_notify list. Reran 
>> the
>> test rig. And have yet to see any KeyErrors.
>
> Have you seen any since the time you reported this?

(I chatted about this with Chris on #zope.)

Since the last report I have not run been running the test rig. I 
Needed my laptop back so I could get on with a few other things. :-(

So I'm going to see if I can show others how to recreate the test rig 
that I'm using here. A trivial site and the script for testing that 
site are attached.

(Reminder - I'm using Zope 2.7.0 here - HEAD just is not ready for such 
games. :-)

SessionRigExtensions.py and sessionrig.zexp make up the site. 
sessionrig.zexp is simply a bunch of external methods (with a folder to 
provide some separation of a standard_error_message that makes use of 
sessions). SessionRigExtensions.py has all the code for the external 
methods.

TestSessionRig.py is the script which attacks zope. It should just work 
as is. (Unless the site you just imported is not found at 
http://localhost:8080/sessionrig. If that's the case then simply edit 
the 'path' in the code.) Don't expect the output of this script to say 
anything conclusive. Instead you should be tailing + greping through 
the event log for things of interest.

Now the knobs that you should tweak to so that you see things of 
interest within minutes/hours rather than days/weeks... (The actual 
values may depend on the hardware that your running this on.)

* etc/zope.conf:

session-timeout-minutes 1
maximum-number-of-session-objects 10000000

* lib/python/Products/Tranience:

PERIOD = 1 #20 # attempt housekeeping every PERIOD seconds
SPARE_BUCKETS = 2 #15 # number of buckets to keep spare

* lib/python/tempstorage/TempoaryStorage.py:

# keep old object revisions for CONFLICT_CACHE_MAXAGE seconds
CONFLICT_CACHE_MAXAGE = 1 #60
# garbage collect conflict cache every CONFLICT_CACHE_GCEVERY seconds
CONFLICT_CACHE_GCEVERY = 1 #60

* lib/python/Products/SiteErrorLog/SiteErrorLog.py:

_rate_restrict_burst = 100000 #5

(I wanted to see repeated errors in the event log. There is probably a 
more obvious way to do that. But that's what I'm using at the moment. 
Took me a while to actually find out this was why I was not seeing all 
the exceptions in the event log. Perhaps things like these should be 
exposed via zconfig? Makes a useful medium to document such knobs to 
promote awareness. As well as allowing people that are pushing zope 
into new corners to have a simpler way to manage these kinds of 
settings.)

* TestSessionRig.py - there are few things that you can tweak within 
here too. (Well the whole script is up for grabs - hack away - I did.)

As the script is it should expose problems with lack of transaction 
isolation for standard_error_message. I do tend to run several 
instances of this script at once. And in my case (as the url suggests) 
I run this on the same host as zope is running on.

However to expose the KeyError thrown from TempoaryStorage I found that 
I did not really need much 'sleepiness' and replaced all the start()'s 
at the end of the script with start(200,1,100,10). (I suspect that you 
don't even need to support cookies in the client for this error. 
Sufficient to hit any URL that simply creates a session object on the 
server. And hit that URL very hard.)

Another effective knob that you may try playing with is the nap 
duration used to slow down a session write:

slowWrite = path + '/write?nap:float=0.5'

hmm... that's probably enough to help someone recreate a KeyError or 
two.

Michael.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: sessionrig.zexp
Type: application/octet-stream
Size: 2447 bytes
Desc: not available
Url : http://mail.zope.org/pipermail/zope-dev/attachments/20040513/40503773/sessionrig.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SessionRigExtensions.py
Type: application/octet-stream
Size: 1111 bytes
Desc: not available
Url : http://mail.zope.org/pipermail/zope-dev/attachments/20040513/40503773/SessionRigExtensions.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: TestSessionRig.py
Type: application/octet-stream
Size: 3120 bytes
Desc: not available
Url : http://mail.zope.org/pipermail/zope-dev/attachments/20040513/40503773/TestSessionRig.obj
-------------- next part --------------