[ZODB-Dev] RE: [Zope] Re: Squid headaches

[Tim Peters]

[Dieter Maurer]
> ...
> In the second packing phase (the so called copying phase),
> the packing thread helds the storage lock most of the time.
> This can make accessing the ZODB impossible by normal operations,
> especially because the "lock_release", "lock_acquire" probably
> does not succeed in giving the lock to a waiting thread.

That decision is made by the platform thread facilities:  the strongest that
can be said in general is that lock_release *offers* to yield to another
thread.  The conditions under which the platform accepts the offer vary a
lot across platforms.

Assuming you're using Linux, there are many reports (in the Python world,
independent of Zope) that most Linux flavors are reluctant to switch threads
given the pthread condition variable + mutex pair Python uses to implement
locks prior to Python 2.3.  When possible, Python 2.3 uses POSIX semaphores
instead to implement locks on pthreads boxes.  Because POSIX semaphores
derived from the real-time world, platforms supporting them are generally
much more willing to switch, often treating the offer to yield more as a
demand to yield.  So an interesting experiment would be to try the same
thing but using Python 2.3b1 instead.  A caution that this can have bad
effects too:  always switching is as irritating to some apps as rarely
switching is to others.  The pragmatics of thread use are a cross-platform,
cross-thread-library, and cross-thread-gimmick mess.