[ZODB-Dev] design issue: job queue is concurrency bottleneck

[John Belmonte]

I finally got through the first round of concurrency testing on my app. 
    Despite being a high-write application, virtually no write conflicts 
occur-- with one major exception.  I have a job queue, which is a BTree
keyed by scheduled job time.  It's the only BTree in my app that doesn't 
have random keys.  Incoming requests to my server tend to generate a few 
jobs, and unfortunately they must be scheduled at various constant time 
periods in the future.  As a result, simultaneous requests generate jobs 
scheduled for approximately the same time, mapping to the same area of 
the BTree, and causing write conflicts.

Short of adding some random offset to scheduled job times, which I feel 
would not be very effective (depends on BTree size, etc.), does anyone 
have ideas for designing around this bottleneck?

-John


-- 
http:// if  ile.org/