May 2000 - Zope-Dev - Zope lists

Re: [Zope-dev] Memory 2.1.4-2.1.6 a.k.a. how to get objects out of the cache
by Eric Sattler 29 May '00

29 May '00

All, I also am having severe problems with memory creep. Our problem exhibits itself by quickly using gobs of memory and requiring a zope restart after about a day and a half at about 100M resident. First our configuration: Core Components: FreeBSD 4.0/FreeBSD 3.4 - on separate machines of course Postgres 6.5.3/Postgres 7.0(fixes some memory leaks) Python 1.5.2 Zope 2.1.4/2.1.6 have tried both pcgi with ZServer Apache w/mod_ssl 1.3.9 and 1.3.12 Products: ZPySQLDA SQLSession GenericUserFolder I have looked at a lot of different things. This includes: Installing the gc(garbage collection) python package - Neil Schemenauer's patches Installing Postgres 7.0(fixes some connection memory leaks) Tracking the cache cleanup inside zope Tracking thread locking inside DB.py, Transaction.py Results: gc python didn't help alleviate the problem Postgres 7.0 instead of 6.5.3 didn't help the problem I do see GenericUserFolder and SQLSession objects with the Control_Panel_Debug screen, and they do not seem to go away. I wrote a simple python script to do nothing more than authenticate (log in) using the GenericUserFolder method docLogin. The memory usage quickly grows out of control. After waiting 15 minutes(my cookie timeout), no decrease in memory usage. The objects are still in the cache also. Restarting zope clears everything up, and starts out nice and clean again. We have a cron job doing this right now. Questions: 1. How does a user's connection resources etc. get cleaned up by Zope after a cookie timeout? Do I need to do this myself? 2. How does a genericuserfolder's set of objects used by an authenticated user get cleaned up. I can't seem to make those objects get reclaimed by the system, even when the user logs off? 3. I am not explicitly removing SQLSession objects. Will my usage counts for these objects remain > 1, thereby never allowing them to get cleaned up, and thereby keeping my usage counts for genericuserfolder's > 1? 4. Since GenericUserFolder inherits from Folder objects, is it possible that the reason my memory grows so fast is that Zope does not release the resources used by a user properly, during an abrupt disconnect? I will assist/help trying different things to try to solve this problem! thanks. eric.

4 4

Z SQL: "optional" isn't?
by Kyler B. Laird 29 May '00

29 May '00

I'd like to build a Z SQL method for *many* people to use to list courses that we offer. I don't want to require that all queries using it understand all of the columns, but I do want to be able to modify the query in many ways. So...after reading http://yyy.zope.org/Documentation/Guides/ZSQL-HTML/ZSQL.1.5.3.html optional A flag indicating if the test is optional. If the test is optional and no value is provided for a variable, or the value provided is an invalid empty string, then no text is inserted. I thought I could just use "optional" with all of my sqltests. select * from course_catalog <dtml-sqlgroup where> <dtml-sqltest subject type=nb optional> <dtml-and> <dtml-sqltest course_nbr type=nb optional> <dtml-and> <dtml-sqltest version type=nb optional> <dtml-and> <dtml-sqltest campus_code type=nb optional> </dtml-sqlgroup> This works only if I provide at least an empty string for all values. If I don't provide a value at all for one of the arguments, I get Error, Bad Request: ['version'] SQL used: Could not render the query template! Traceback (innermost last): File /data/www/Zope/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 224, in publish_module File /data/www/Zope/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 189, in publish File /data/www/Zope/Zope-2.2.0a1-src/lib/python/Zope/__init__.py, line 221, in zpublisher_exception_hook (Object: list_courses) File /data/www/Zope/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 175, in publish File /data/www/Zope/Zope-2.2.0a1-src/lib/python/ZPublisher/mapply.py, line 160, in mapply (Object: manage_test) File /data/www/Zope/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 112, in call_object (Object: manage_test) File /data/www/Zope/Zope-2.2.0a1-src/lib/python/Shared/DC/ZRDB/DA.py, line 347, in manage_test (Object: list_courses) File /data/www/Zope/Zope-2.2.0a1-src/lib/python/Shared/DC/ZRDB/DA.py, line 324, in manage_test (Object: list_courses) File /data/www/Zope/Zope-2.2.0a1-src/lib/python/Shared/DC/ZRDB/DA.py, line 426, in __call__ (Object: list_courses) File /data/www/Zope/Zope-2.2.0a1-src/lib/python/Shared/DC/ZRDB/Aqueduct.py, line 150, in _argdata (Object: list_courses) Bad Request: (see above) Looking at Aqueduct.py, I don't understand why it's not catching the "optional" arg. Using my limited Python knowledge, I commented out lines 149 and 150 of Aqueduct.py and now it works great. Once again, I'm glad that I can add these awful kludges instead of waiting for the vendor, but this is something that I hope is fixed soon. --kyler

3 4

New GUF (1.2.3)
by Stuart 'Zen' Bishop 29 May '00

29 May '00

I think I've fixed the memory leak identified over the weekend in GUF. This leak may also have caused {SQL,FS}Session to leak through no fault of their own. Could people who are experiencing the problem try out the new version and let me/zope-dev know the results. I'll announce the release on zope.org soonish, giving the guinee^h^h^h^h^h^hzope-dev members a chance to give it a poke. -- Stuart Bishop Work: zen(a)cs.rmit.edu.au Senior Systems Alchemist Play: zen(a)shangri-la.dropbear.id.au Computer Science, RMIT University

1 0

Data.fs corruption (reproducable)
by Dr. Peer Griebel 29 May '00

29 May '00

Hi, I just discovered that my Data.fs get's periodically crashed. The symptom is that one single external method in Zope will be presented as a folder. I can't edit the method nor delete it. In the output I can find many messages like ZODB Couldn't load state for '\000\000\000\000\000\000\000\263' (The numbers are just guessed since I don't have them currently.) After the first occurence of the problem I copied all of my projects into a clean new Data.fs but the problem came back instantly. Now I could determine how to prevent the problem: I have to change the implementation of my external method. The external method is mostly a copy of the method thousands_commas in DT_var.py. All I did is to replace dots by commas and vice versa to implement the German representation of numbers: def FormatPreis( v, n=2, thou=regex.compile( "$[0-9]$$[0-9][0-9][0-9]\([,.]\|$$\)").search): v = "%.*f" % ( n, v ) vl = string.split(v,'.') if not vl: return v v=vl[0] del vl[0] if vl: s = ',' + string.join(vl,',') else: s = '' l=thou(v) while l >= 0: v=v[:l+1]+'.'+v[l+1:] l=thou(v) return v+s When this method gets called by some of my dtml-documents Data.fs gets crashed. (Therefore I started to make backups every few minutes.) Since I replaced the method above by a very simple method def FormatPreis( v, n=2 ): return str(v) my data.fs stays intact! So what's happening here??? I hope there is no major problem within Zope. But I would like to see a hint what's going on. Thank you for your help! -- connection reset by Peer _______________________________________________________________________ Dr. Peer Griebel Tel. +49 7581 4831 23 Geschäftsführer Fax. +49 7581 4831 11 Knoll Informationssysteme GmbH http://www.knoll-is.de Dreiköniggasse 17 mailto:peer@knoll-is.de 88348 Saulgau privat: mailto:peer.griebel@gmx.de

2 2

Zope 2.1.6 DateTime bug: daylight saving time not handled -- with patch
by Dieter Maurer 28 May '00

28 May '00

The Zope 2.1.6 DateTime module does not recognize daylight saving time. Patch attached. Dieter

2 1

Really buggy digest software
by Dieter Maurer 28 May '00

28 May '00

Dear administrators, the mailing list digests are really buggy. I have reported some problems some days ago. Today, I ran against a new problem: I got "Zope-Dev digest, Vol 1 #543 - 20 msgs" 6 times and "Zope digest, Vol 1 #801 - 57 msgs" twice. Maybe, you should consider replacing this software by another mailing list management system. Dieter

1 0

use ODBC to access other database
by Thomas Weholt 28 May '00

28 May '00

Hi, I have a Interbase-database I want to access from Zope, but cannot seem to find any Database-adapter-products etc. anywhere. Could I use a ODBC-adapter in Zope to connect to this database, or on some other manner connect to this database using SQL-methods in Zope-pages? Does anybody know if there are a Interbase-databaseadapter in development? Interbase is open-source, free etc. and allready an established product so it would be strange if it wasn`t supported by Zope, at least in the future. Thomas

2 1

newbie question: parameter passing for ZSQL Object
by zope 28 May '00

28 May '00

Hi. With some values that I got from a formular, I wan't to do some calculation stuff, and insert the calculated values into a database. My Z SQL method'insertItem' looks like Arguments: param1, param2, param3, param4 INSERT INTO plan (field1, field2, field3, field4) VALUES (<dtml-var param1>, <dtml-var param2>, <dtml-var param3>, <dtml-var param4>) (parameters should be integer values) Can anybody tell me how I can call this method and pass parameters from my DTML method? With <dtml-call insertItem>, Zope gives me following error message: Zope has encountered an error while publishing this resource. Error Type: Bad Request Error Value: ['param1', 'param2', 'param3', 'param4'] I was checking then "ZSQL Methods User's Guide" but, they don't give an example :-( Please help. Any idea? Thanks, Marc.

3 2

Zope 2.1.6 ZCatalog: minor bug, missing "find" import -- with patch
by Dieter Maurer 28 May '00

28 May '00

I just found a minor bug in Zope 2.1.6's ZCatalog (distribution). "ZCatalog.py" does not import "find". If the "containing" field contains a value, a NameError exception is raised. Patch appended. Dieter

1 0

Problems with RackMountable
by Itamar Shtull-Trauring 28 May '00

28 May '00

I'm adding some RackMountable objects (subclassed from RackMountable and SimpleItem.Item) and when I try to access them via URL I get an AttributeError like they don't have attribute __of__: Traceback (innermost last): File /hd1/zope/lib/python/ZPublisher/Publish.py, line 214, in publish_module File /hd1/zope/lib/python/ZPublisher/Publish.py, line 179, in publish File /hd1/zope/lib/python/Zope/__init__.py, line 202, in zpublisher_exception_hook (Object: ElementWithAttributes) File /hd1/zope/lib/python/ZPublisher/Publish.py, line 151, in publish File /hd1/zope/lib/python/ZPublisher/BaseRequest.py, line 308, in traverse File /hd1/zope/lib/python/Products/ZPatterns/Specialists.py, line 27, in __bobo_traverse__ (Object: SheetProviderContainer) File /hd1/zope/lib/python/Products/ZPatterns/Specialists.py, line 39, in getItem (Object: SheetProviderContainer) File /hd1/zope/lib/python/Products/ZPatterns/Rack.py, line 515, in __get_attr__ (Object: cheese4) AttributeError: __of__ What am I doing wrong here? -- Itamar S.T. itamar(a)maxnm.com

1 0