Hi folks, like Sven & Philip i have to report similar strange happenings on our server Profile: RH Linux 7.0 on a Compaq Alpha 1Gig Ram Zope 2.3.0 Postgres 7.0.3 w_pcgi configuration with Zope.cgi and Apache psycopg Postgres DA we tried also PoPy as DA connector but still the same result ... it seems to me, if all those cases are related, that RH7.0 and Postgres are used on all machines where this phenomen occurs. we have on this server an average hitcount of 3000 hits per hour so i suppose you can imagine that it is not really funny to restart a production server every hour. zope runs under 6 python threads ... 7119 nobody 1 0 27848 27M 3440 S 0.2 1.3 0:12 python 7124 nobody 0 0 27848 27M 3440 S 0.0 1.3 0:00 python 7125 nobody 10 0 27848 27M 3440 S 5.2 1.3 0:34 python 7126 nobody 0 0 27848 27M 3440 S 0.0 1.3 0:27 python 7127 nobody 0 0 27848 27M 3440 S 0.0 1.3 0:34 python 7128 nobody 0 0 27848 27M 3440 S 0.0 1.3 0:25 python 7143 postgres 5 0 6200 6200 4240 S 2.9 0.3 3:01 postmaster 8455 postgres 0 0 5840 5840 4136 S 0.0 0.2 0:20 postmaster 7118 root 0 0 5056 5056 2216 S 0.0 0.2 0:00 python 2511 apache 1 0 4344 4344 3896 S 0.0 0.2 0:00 httpd 2998 apache 3 0 4344 4344 3896 S 0.0 0.2 0:00 httpd 4755 apache 0 0 4344 4344 3896 S 0.0 0.2 0:00 httpd 6600 apache 0 0 4344 4344 3896 S 0.0 0.2 0:00 httpd 3573 apache 2 0 4336 4336 3896 S 0.0 0.2 0:00 httpd 4327 apache 1 0 4336 4336 3896 S 0.0 0.2 0:00 httpd 8726 apache 0 0 4336 4336 3896 S 0.0 0.2 0:00 httpd 6231 apache 0 0 4320 4320 3880 S 0.0 0.2 0:00 httpd 7414 apache 2 0 4320 4320 3880 S 0.0 0.2 0:00 httpd 7780 apache 4 0 4320 4320 3880 S 0.0 .................. the memory usage of the python threads grows slowly but constantly until zope dies .... has anyone a hint? What would be the best method to locate the memory leak? Thank you Francisco Webber
Francisco Eduardo Webber wrote:
Hi folks, like Sven & Philip i have to report similar strange happenings on our server
Profile: RH Linux 7.0 on a Compaq Alpha 1Gig Ram Zope 2.3.0 Postgres 7.0.3 w_pcgi configuration with Zope.cgi and Apache psycopg Postgres DA
My Profile: SuSE Linux 6.3 256MB Ram Zope 2.3.1b1 Postgres 7.0.2 ZServer psycopg,ZPyGreSQLDA After a long psycopg install session last night I give up in frustration and switch back to ZPyGreSQLDA. Directly after installing psycopg the DA seems to work. I did some performance tests. If the connection is already established psycopg is not faster then ZPyGreSQLDA. But there is the same issue with ZSQLMethod's 'Maximum rows to retrieve' property. IMHO this prop is simply useless because you get a definitely wrong result if you have more rows in your db then configured on the DA. I can't see the benefit of a fast but wrong query result ... ZPyGreSQLDA simply ignores this prop ;-) How ever ... after restarting the server I was in trouble. ZServer died on the first query I made on psycopg. I started postmaster on debug level 3 but could not figure out the problem :( On psql I could do queries with no problem. But both psycopg AND ZPyGreSQLDA did not work. After uninstalling psycopg and restarting Zope ZPyGreSQLDA works fine.
we tried also PoPy as DA connector but still the same result ... it seems to me, if all those cases are related, that RH7.0 and Postgres are used on all machines where this phenomen occurs.
Both psycopg and PoPy seems to have a problem on initialisation.
we have on this server an average hitcount of 3000 hits per hour so i suppose you can imagine that it is not really funny to restart a production server every hour.
I saw the discussion on this list about the various postgres DA's but we have ZPyGreSQLDA on a intranet ERP system for more than one year with no problems.
has anyone a hint? To use ZPyGreSQLDA on Zope 2.3 you have to patch addConnectionForm method in DA.py
addConnectionForm=HTMLFile('Shared/DC/ZRDB/dtml/connectionAdd',defau.... ^^^^ -- _______________________________________________________________________ Andreas Heckel andreas@easyleading.org LINUX is like a wigwam...no gates...no windows and an apache inside ;-)
Hello all, i followed (started to do so ...) to clean the code from HTTP calls on python functions passing more than 9 variables. We pack the variables all in a string to only pass ONE argument. Result after a few major modifications ... the growthrate of the memory leak is much slower ... even seems to become negative sometimes ... but in an average the leak still grows .... Andreas, concerning the ZpyGreSQLDA adapter ... isnt that one single threaded? That would be very limiting in our case ... dieter, i didnt find the reference on "SQL-Alias" in the recent mail archive ... (or am i blind ;-) and other question... has anyone using the 2.3.1b2 > versions ever had those memory leak problems? Greetings Francisco
Francisco Eduardo Webber wrote:
Andreas, concerning the ZpyGreSQLDA adapter ... isnt that one single threaded? That would be very limiting in our case ... Yes :( But 3000 hits per 3600 seconds on a Compaq Alpha with 1GB RAM should be ok. One of my queries that returns ~21000 rows on my Celeron 500 256MB RAM ATA33 EIDE takes ~0.446s. And if I use LIMIT/OFFSET to get only 20 rows at one time it takes ~0.006 seconds :-)
-- _______________________________________________________________________ Andreas Heckel andreas@easyleading.org LINUX is like a wigwam...no gates...no windows and an apache inside ;-)
we tried also PoPy as DA connector but still the same result ... it seems to me, if all those cases are related, that RH7.0 and Postgres are used on all machines where this phenomen occurs.
yes, and sometimes, it's mysql that goes down (and not zope) it makes either : IndexError or OperationalError or KeyError then we reboot mysql (through webmin) and a couple of seconds later it works again (that would of course comfort the idea that this is caused by a memory leak). being not *that* fluent in linux, could someone give me the command that would give me some interresting output (i'm usign ps aux to check what is running and it gives me the amount of memory used by the prog, is that enough ?) most of the time it crashes silently, forcing me to killal python then restart zope, but from time to time (once a day?) the process entirely dissappears...
we have on this server an average hitcount of 3000 hits per hour so i suppose you can imagine that it is not really funny to restart a production server every hour.
even worse, it's our *real* server that's going down every 2-3 hours (but we make about 4 times more hits/hour (but we use less database - so maybe that's linked.) $ven
participants (3)
-
Andreas Heckel -
Francisco Eduardo Webber -
Junk