[Zope] Multithreading sessions

[Jon Emmons]

Carol,

 

As a quick fix, we have created a stand alone program that does the database
access.

So from the python external method (zope) that retrieves data, instead of
making the connections to the DB directly it now calls os.popen("python
getdata.py").

 

This forces the launch of a separate python interpreter for the duration of
the query.

 

Unfortunately, we will suffer the overhead of doing this for each DB call,
but that is far better than denying service to all other sessions.

 

We are already in the process of moving to MySQL for our databases, and the
MySQL adapter does not appear to exhibit this behavior.

 

Jon Emmons

 

  _____  

From: Carol Ludwig [mailto:csl at med.unc.edu] 
Sent: Wednesday, June 04, 2008 10:58 AM
To: Jon Emmons
Cc: 'Dieter Maurer'; zope at zope.org
Subject: Re: RE: [Zope] Multithreading sessions

 

We are using Plone and Zope against a legacy Ingres database, and I found a
similar problem
using the adapter provided.  I found that the C library modules (open
source) actually contained
calls to make the adapter work as we wanted, much like our other web
interfaces which maintain
the db and user permissions info as a "connection", but close the actual
query after each.  I had
to modify the Python Zope product to set "autocommit" and call the desired
close in order to get 
the behaviour we were looking for -- keeping the connection, but releasing
each session so that
 the DB backend does not register an active session between actual queries.

That's open source.  Code terminology for "re-write it".

----- Original Message -----
From: Jon Emmons <jon.emmons at earthwavetech.com>
Date: Wednesday, June 4, 2008 8:50
Subject: RE: [Zope] Multithreading sessions
To: 'Dieter Maurer' <dieter at handshake.de>
Cc: zope at zope.org

> Dieter,
> 
> Thanks for this input.  Your comments here turned out to be 
> dead on.  The
> problem is the GIL.
> 
> We are currently running Sybase databases using the python 
> Sybase-0.37
> module for data access.
> 
> This module will set the GIL.  So if the query takes any 
> time at all, every
> other session will freeze.  It does release it when the 
> connection is
> closed, but if the query takes awhile it effectively denies 
> service to every
> other session.
> 
> I am just learning about this, but my initial inquiries suggest 
> that the
> only way to achieve true concurrency using a language like 
> python is to
> launch multiple interpreters.
> 
> I don't yet have the solution to my problem, but at least now I 
> know what
> the problem is, and that is half the battle.
> 
> Thanks again, and thanks to everyone's input, it has helped immensely.
> 
> Jon Emmons
> 
> -----Original Message-----
> From: Dieter Maurer [mailto:dieter at handshake.de] 
> Sent: Wednesday, May 28, 2008 2:27 PM
> To: Jon Emmons
> Cc: zope at zope.org
> Subject: Re: [Zope] Multithreading sessions
> 
> Jon Emmons wrote at 2008-5-23 08:58 -0400:
> > ...
> >I am running zope 2.9.4 and have observed that it will not 
> simultaneously>serve pages to my users.
> 
> Usually, it does.
> 
> I have seen database adapter packages (an old "psycopg" version, 
> to be
> precise) that forgot to release the GIL for some operations.
> Then Zope freezes while these operations are executed.
> 
> 
> A surprising report of a similar kind was: while Zope's embedded
> profiler is in use, Zope effectively runs in single thread mode (such
> that other threads do not distort the timing results).
> Probably not your problem ... but who knows.
> 
> 
> Further potential reasons (for almost surely not responsible for your
> current problem:
> 
>   Expensive operations implemented in "C" (such as 
> operations on huge
>   strings)
> 
>   Creating excessive amounts of objects (causing lots
>   of generation 2 garbage collections which hold the GIL)
> 
> 
> 
> -- 
> Dieter
> 
> _______________________________________________
> Zope maillist  -  Zope at zope.org
> http://mail.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists - 
>  http://mail.zope.org/mailman/listinfo/zope-announce
>  http://mail.zope.org/mailman/listinfo/zope-dev )
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.zope.org/pipermail/zope/attachments/20080604/57eb10f6/attachment.html