[Zope-DB] Re: SQL Relay

[Kent Hoxsey]

Marc-Andre Lemburg wrote:

> In that case the Python interpreter lock looks like
> the most probable cause. Note that the PIL blocks the Python interpreter
> as a whole - regardless of whether other threads continue to run or not;
> Python will simply not continue to evaluate any more byte-codes until
> the PIL is released.

Hmm. That would be a significant change to my worldview. But does it square
with experience? Let's see:

- Umberto has written extensively about the java and python tests his team
wrote to investigate possible threading issues in python. There is an outside
chance his python code used seperate python interpreters (and therefore
global interpreter locks), but since the intent was to compare python to java,
I actually doubt that. However, Umberto can verify this point for us:

Umberto, is it possible your multi-threading python database access program
avoided the global interpreter lock problem?

- SQLRelay talks to a connection server over standard sockets. In order
to get a feel for what's going on here, we need to look into the c code,
which is a bit outside of my normal skills. But luckily, there is documentation
in the python headers for how to do this stuff effectively.

Canonical use, from the instructions of include/python/ceval.h:

/* Interface for threads.

   A module that plans to do a blocking system call (or something else
   that lasts a long time and doesn't touch Python data) can allow other
   threads to run as follows:

	...preparations here...
	Py_BEGIN_ALLOW_THREADS
	...blocking system call here...
	Py_END_ALLOW_THREADS
	...interpret result here...


The code from SQLRelay (specifically, CSQLRelay.c):

static PyObject *sendQuery(PyObject *self, PyObject *args) {
  char *sqlString;
  long sqlrcur;
  int rc;
  if (!PyArg_ParseTuple(args, "ls", &sqlrcur, &sqlString))
    return NULL;
  Py_BEGIN_ALLOW_THREADS
  rc=((sqlrcursor *)sqlrcur)->sendQuery(sqlString);
  Py_END_ALLOW_THREADS
  return Py_BuildValue("i", rc);
}

Looks ok to me, but I'm not an expert. Any experts have observations?

- DCOracle: Hmm, whole buncha c code here, but the place to look seems
to be dco2.c, in the function definition for Cursor_execute:

	Py_BEGIN_ALLOW_THREADS
	
	status = OCIStmtExecute(self->sc->svchp, self->stmtp, self->errhp,
		iters, rowoff, NULL, NULL, OCI_DEFAULT);

	Py_END_ALLOW_THREADS


which also seems to fit with the documentation.

Based on this quick survey of the two APIs I've been using to generate the
problem, it would seem that the global interpreter lock is not being triggered
by the database APIs themselves.

I'm still in the process of chasing demons with gdb, and will report progress
once I've made some. Anybody with additional insight, please jump in.

Kent