[ZODB-Dev] design issue: job queue is concurrency bottleneck
John Belmonte
john at neggie.net
Thu Mar 4 10:35:34 EST 2004
Christian Robottom Reis wrote:
> On Thu, Mar 04, 2004 at 10:11:51AM -0500, John Belmonte wrote:
>
>>>Have you considered trying to add the key, catching ConflictError, and
>>>trying again adding a small time offset?
>>
>>My conflict errors are not due to writing to the same keys. From my
>>limited understanding of BTrees, adding keys may cause buckets to be
>>shuffled around, resulting in a write error.
>
> Well, the same suggestion applies: catch the ConflictError, sync() your
> current connection (or abort() your current transaction), and try again?
Yes, I'm aborting and running the transaction again. However, I'm
trying to eliminate write conflicts because I believe, given a
high-write application, that's the only scalable solution. For example,
with my very slow development machine acting as client and server (a
nice worst-case test), and with two simultaneous requests, one client
must retry up to three times due to the job queue conflict. With three
simultaneous requests, up to six times. I haven't tried four requests
because I don't want to find out that it's an exponential progression :-).
-John
--
http:// if ile.org/
More information about the ZODB-Dev
mailing list