Product refresh causes database connections leaking
I'm developing a product (based on SimpleUserFolder), which adds a folder with some ZSQL methods to itself on instanciation. The ZSQL methods use a database adapter (Z Psyco DA 1.1.3, symptoms occur also with older versions). After some time things went awry, because the DA had lost its connection to the database and couldn't reconnect, because the max connections postgres would allow were used. Now I think I found the cause, because everytime I refresh my product, and the ZSQL Methods are used, the number of connections to postgres grows by 4. Anybody knows what exactly is to blame, zope's refresh procedure or the database adapter? cheers, oliver
Oliver Bleutgen writes:
I'm developing a product (based on SimpleUserFolder), which adds a folder with some ZSQL methods to itself on instanciation. The ZSQL methods use a database adapter (Z Psyco DA 1.1.3, symptoms occur also with older versions).
After some time things went awry, because the DA had lost its connection to the database and couldn't reconnect, because the max connections postgres would allow were used.
Now I think I found the cause, because everytime I refresh my product, and the ZSQL Methods are used, the number of connections to postgres grows by 4.
Anybody knows what exactly is to blame, zope's refresh procedure or the database adapter?
Maybe it is Your code which is to blame, who knows ;-) It seems the SQL connections were not garbage collected correctly, or if they were, the corresponding connections were not closed properly. It is a little hard to say what is going on without details how and where the connections are created. Anyway I have heard rumours, that the "refresh" feature is not considered the most stable thingy in Zope, but more a developer convenience hack. If You restart Your development server every forth time You usually would do a refresh, You save Yourself a longer debugging session. just my 2 cents clemens
Clemens Robbenhaar wrote:
It is a little hard to say what is going on without details how and where the connections are created. Anyway I have heard rumours, that the "refresh" feature is not considered the most stable thingy in Zope, but more a developer convenience hack. If You restart Your development server every forth time You usually would do a refresh, You save Yourself a longer debugging session.
That's about right. The refresh functionality uses features of ZODB and Python that haven't been perfected, and the result is occasional surprises. My rule is that if there's no really obvious explanation (i.e. about 5 seconds) for a bug that appears after a refresh, I restart Zope. The most common refresh-related bug is "AttributeError: None has no attribute <foo>". It happens because an old class instance or function is still alive. If that happens often, fix up the refresh dependencies to ensure that derived products are always refreshed after a base product. Shane
Shane Hathaway wrote:
That's about right. The refresh functionality uses features of ZODB and Python that haven't been perfected, and the result is occasional surprises. My rule is that if there's no really obvious explanation (i.e. about 5 seconds) for a bug that appears after a refresh, I restart Zope.
The most common refresh-related bug is "AttributeError: None has no attribute <foo>". It happens because an old class instance or function is still alive. If that happens often, fix up the refresh dependencies to ensure that derived products are always refreshed after a base product.
But in this case it's not a problem with the product. The DA is _outside_ the folderish product, the ZSQL Methods are inside (in a stock zope folder on their own). So everything related to the SQL stuff is stock zope stuff. Refreshing causes the number of connections from zope to the postgres to grow (from 4 to 8 in the first time). That's why I suspected there's something wrong either with the DA or with the refresh functionality which has nothing to do with the ZODB. cheers, oliver
On Thu, 8 May 2003, Oliver Bleutgen wrote:
Refreshing causes the number of connections from zope to the postgres to grow (from 4 to 8 in the first time). That's why I suspected there's something wrong either with the DA or with the refresh functionality which has nothing to do with the ZODB.
The refresh functionality *does* have to do with ZODB. Specifically, Refresh attempts to clear the cache, but in Zope 2.6 it does not succeed in freeing objects from the cache. That is why it's leaking connections. In Zope 2.7 it will hopefully succeed in freeing the cached objects. Shane
Shane Hathaway wrote:
On Thu, 8 May 2003, Oliver Bleutgen wrote:
Refreshing causes the number of connections from zope to the postgres to grow (from 4 to 8 in the first time). That's why I suspected there's something wrong either with the DA or with the refresh functionality which has nothing to do with the ZODB.
The refresh functionality *does* have to do with ZODB.
Erm, yes, this sentence shouldn't express that, we need parenthesis also in natural languages, maybe.
Specifically, Refresh attempts to clear the cache, but in Zope 2.6 it does not succeed in freeing objects from the cache. That is why it's leaking connections. In Zope 2.7 it will hopefully succeed in freeing the cached objects.
Ok, thanks for the explanation, I see this problem is known. cheers, oliver
participants (3)
-
Clemens Robbenhaar -
Oliver Bleutgen -
Shane Hathaway