On Fri, Sep 19, 2008 at 9:23 AM, Marco Bizzarri <marco.bizzarri@gmail.com> wrote:
Hi all.
I'm working on an application which uses Zope (2.8, at the moment) and ZPsycopgDA (toghter with a number of other products).
While writing an acceptance test, I encountered a strange problem: the test locks up.
A further investigation shown that there were two connections at the database; one of them was not committed, the other one was blocked waiting for the other to commit.
I therefore used the pdb in order to stop the execution of the test inside the connect method of the ZPsycopgDA.DA. Once I had that breakpoint, I was able to get the logs of the two transactions on the database, and I had the confirmation that indeed there were two different transactions.
So, I wondered what could possibily happen, I mean why during a test there could be a second connect to the database.
I issued a "bt" to see the stack of calls leading to the connect, and what I could see was that the coonect was called inside the __setstate__ method of Shared/DC/ZRDB/Connection.py.
I assume therefore that the ZPsycopgDA object has been "ghostified", during the transaction. But this "assumption" is not supported by any evidence. In particular, it is not supported by my knowledge of the internal behaviour of ZODB on objects during a single transaction.
Can anyone provide suggestion on this topic?
Regards Marco -- Marco Bizzarri http://notenotturne.blogspot.com/ http://iliveinpisa.blogspot.com/
I did further investigation on the topic, and I think I've pinned the problem. I don't know the solution, but I can reproduce the problem with a small sample. Here is the sample: import os import sys import unittest if __name__ == '__main__': execfile(os.path.join(sys.path[0], '../framework.py')) from Testing import ZopeTestCase from OFS import Image from Products.ZPsycopgDA.DA import manage_addZPsycopgConnection from Products.ZSQLMethods import SQL class DoubleTransactionTest(ZopeTestCase.ZopeTestCase): def _add_big_image(self, value, data): Image.manage_addFile(self.app, "f%06s" % value, data , "a title") def test_showdouble(self): manage_addZPsycopgConnection(self.app, "db_connection", "", "host=localhost user=postgres dbname=template1") self.app._setObject('sql', SQL.SQL("sql", "", "db_connection", "", "select * from pg_tables")) self.app.sql() data = "*" * (1 << 20) for x in range(1000): self._add_big_image(x, data) print "Added %s " % x self.app.sql() if __name__ == '__main__': unittest.main() I'm doing three things here: - creating a db connection - making a query to the db (this causes a transaction to begin) - creating a lot of "big" files (expecially, larger than 2 * 2 ^ 16 *) - making another query to the db; Once I create a big file I fall into the following branch inside the OFS.Image._read_data if size <= 2*n: seek(0) if size < n: return read(size), size return Pdata(read(size)), size # Make sure we have an _p_jar, even if we are a new object, by # doing a sub-transaction commit. transaction.savepoint(optimistic=True) This causes, at the end, to call the ZODB.Connection.savepoint which, just before returning, calls a cacheGC to be called, which, I'm afraid, causes the db_connection to be "sent" out of the cache itself, thus leaving it without the _v_ attributes. Hope this can help in giving suggestions. Regards Marco -- Marco Bizzarri http://notenotturne.blogspot.com/ http://iliveinpisa.blogspot.com/