[ZODB-Dev] Sharing (persisted) strings between threads

Malthe Borch mborch at gmail.com
Wed Dec 8 08:08:17 EST 2010


On 8 December 2010 14:03, Hanno Schlichting <hanno at hannosch.eu> wrote:
> On Wed, Dec 8, 2010 at 11:06 AM, Malthe Borch <mborch at gmail.com> wrote:
>> With 20 active threads, each having rendered the Plone 4 front page,
>> this approach reduced the memory usage with 70 MB.
>
> Did you measure throughput of the system? In the benchmarks I've seen
> threads numbers of 3 or above will perform worse than one or two
> threads. At least with the GIL implementation up to Python 2.6 you get
> much worse performance the more threads you have on multicore systems.
> There's good explanations of the behavior done by David Beazley at
> http://www.dabeaz.com/blog.html.

I should have mentioned: I only chose 20 threads such that I might see
a noticeable difference in memory usage.

> In default Plone 4 we have two threads per instance. If you have more
> than a single ZEO instance you should reduce the thread number to one.
> We also set a default Python checkinterval of 1000 (instructions),
> which prevents thread switching for long stretches of time to counter
> the GIL in the two thread case.

I agree. As I mentioned to Jim, there's a classic case of running two
threads. However, there's also a case of running more if you expected
to be swapping data in and out at regular intervals.

> So while sharing data between threads might sound interesting, it's
> not of much help in Python.

I think it's worthwhile even sharing immutable data between two
threads, if it's a relatively straight-forward procedure. I think my
initial investigation shows that it is.

\malthe


More information about the ZODB-Dev mailing list