May 2000 - Zope-Dev - Zope lists

ZServerSSL trouble
by Ivan Raikov 31 May '00

31 May '00

Greetings, I just installed ZServerSSL, but when I try to access the https server, I get the following: Traceback (innermost last): File "/home/ivan/src/Zope-2.1.6-src/z2s.py", line 681, in ? asyncore.loop() File "/home/ivan/src/Zope-2.1.6-src/ZServer/medusa/asyncore.py", line 115, in loop poll_fun (timeout) File "/home/ivan/src/Zope-2.1.6-src/ZServer/medusa/asyncore.py", line 62, in poll (r,w,e) = select.select (r,w,e, timeout) ValueError: filedescriptor out of range in select() I don't know enough about Zope innards, so I was wondering if someone can explain when such a situation can occur. I also noticed that the lists (or tuples) r and w get their values from an object called `socket_map', so I'd like to know what the purpose of this object is, and where it is initialized. Regards, Ivan Raikov

1 0

ZODB article draft
by Andrew M. Kuchling 31 May '00

31 May '00

I've just put up the first draft of a little introductory article about ZODB and ZEO. Please check it out and offer comments. Comments from inexperienced people are particularly welcome; is it a decent introduction? Which bits are confusing? http://starship.python.net/crew/amk/python/writing/zodb-zeo.html BTW, I'm looking for a good place to send this article -- byte.com? Oreillynet/python? -- please let me know your suggestions through private e-mail. --amk

1 0

Traversal Stuff
by Chris Withers 31 May '00

31 May '00

Hi, I just remembered, something I really miss from my Mason days are dhandlers. What this offers is having a URL like: http://mysite.com/folder/object/some/parameters processed by the 'object' object with a parameter of '/some/parameters' or better still a 'URL objects' list as a parameter. This sounds like Traversal Interface stuff so I thought now would be a good time to mention it given the traversal interface is being re-done. What do people think? cheers, Chris PS: I'm off to dump this into the interfaces Wiki right now... ;-) PPS: I know ZSQL methods can do something similar so it must be possible ...

2 1

Re: [Zope-dev] Zope 2.2.0 alpha 1 released...
by Anthony Baxter 31 May '00

31 May '00

>>> Brian Lloyd wrote > To me, the more hairy issue is what if Joe *does* exist in the > different Zope installation, and you *do* want Joe to continue > to have ownership? Currently, you must either arrange for Joe > to do the import (which will give him ownership directly), or > import it and use an external method to assign ownership (which > is a pain). One thing we've thought of is that perhaps superuser > (and only superuser) could be able to assign ownership through > a web interface, which could make this sort of thing a bit > easier. Hm. How about some sort of toggle that allows an import to maintain ownership from the import file? Course, you need to specify who is _allowed_ to set that toggle, or else anyone who can import can set up any ownership that they like. Anthony -- Anthony Baxter <anthony(a)interlink.com.au> It's never too late to have a happy childhood.

6 6

Re: [zope-xml] XMLDocument beta release 3
by Uche Ogbuji 31 May '00

31 May '00

> This seems still to be rather linux-centric. Now with the 4DOM binaries for > windows released, would there be a way to use this on win32 without jumping > to a string of hoops? Now that we have "official" Windows Binaries for all the 4Suite tools I think the greatest barrier is dhaving to patch Zope. This will likely be addressed by providing a CVS branch for the XML integration and incorporating it all into the existing cross-platform compatible structure. Hopefully we'll have this sorted out very quickly, but at least we have eliminated 75% of the hoops in question. We're working on the rest. Thanks. -- Uche Ogbuji Principal Consultant uche.ogbuji(a)fourthought.com +01 303 583 9900 x 101 Fourthought, Inc. http://Fourthought.com 4735 East Walnut St, Ste. C, Boulder, CO 80301-2537, USA Software-engineering, knowledge-management, XML, CORBA, Linux, Python

1 0

Re: [Zope-dev] Memory 2.1.4-2.1.6 a.k.a. how to get objects out of the cache
by Jason Spisak 31 May '00

31 May '00

Eric Sattler: > Jason, > > I don't have time right now to post this to the zope > newsgroup, but we are bleeding profusely, and could > use any insight you might have... > Unfortunately just before recieving your message, I fired another one out because I have run out of tests. :( > > I also am having severe problems with memory creep. Our problem > exhibits itself by quickly using gobs of memory and eventually requiring > a > zope restart. > > First our configuration: > > Core Components: > > FreeBSD 4.0/FreeBSD 3.4 - on separate machines of course > Postgres 6.5.3/Postgres 7.0(fixes some memory leaks) > Python 1.5.2 > Zope 2.1.4/2.1.6 > Apache w/mod_ssl 1.3.9 and 1.3.12 > > Products: > > ZPySQLDA > SQLSession > GenericUserFolder > There was recently a post that detained a leak in a database product. I can't remeber if it was SQLSession, or what. > I have looked at a lot of different things. This includes: > > Installing the gc(garbage collection) python package > Installing Postgres 7.0(fixes some connection memory leaks) > Tracking the cache cleanup inside zope > Tracking thread locking inside DB.py, Transaction.py > > Results: > > gc python didn't help alleviate the problem > Postgres 7.0 instead of 6.5.3 didn't help the problem > > I do see GenericUserFolder and SQLSession objects with the > Control_Panel_Debug screen, and they do not seem to go away. I wrote > a simple python script to do nothing more than authenticate (log in) > using the GenericUserFolder method docLogin. The memory usage quickly > grows out of control. After waiting 15 minutes(my cookie timeout), > no decrease in memory usage. The objects are still in the cache also. This is consistant with what I remember seeing on the list. I don't use any external databases, so my situation is really an argument for their being something amiss in Zope or Python. > > Questions: > > 1. How does a user's connection resources etc. get cleaned up > by Zope after a cookie timeout? Do I need to do this myself? > I don't use cookie here. > 2. How does a genericuserfolder's set of objects used by an > > authenticated user get cleaned up. I can't seem to make those > objects get reclaimed by the system, even when the user logs > off? is there a magic cache setting somewhere? > Or GUF. > 3. I am not explicitly removing SQLSession objects. Will my usage > counts for these objects remain > 1, thereby never allowing them > to get cleaned up, and thereby keeping my usage counts for > genericuserfolder's > 1? > > 4. Since GenericUserFolder inherits from Folder objects, is it > possible that the reason my memory grows so fast is that Zope > does not release the resources used by a user properly, during > an abrupt disconnect? That's funny that I am experienceing the same symptoms as you, but my installation is totally different. I'm not using GUF, or any DA's, and I'm running on RH 6.1. We really need to figure this out as a community. I am willing to try just about anything short of shipping my machine somewhere off premisis. ;) Bledding on the floor, Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.

3 4

Put Zope objects into a ZODB
by Petr Knapek 31 May '00

31 May '00

Hi all, I would like to use ZODB3 as an object DB to store my objects. I know How-To by Michael Pelletier on this topic. So that is to store objects in ZODB out of Zope. I prefer to store objects which are subclassed of Zope objects (especially Folder and ZClass - not multiple) because in future I would like to import this my ZODB into a Zope ZODB as its part into a folder. Could anybody help me where to find any information, documentation ... My task is to use ZODB as object database to storage objects. Containers, subclassed from Zope Folder which will store other objects sublassed from ZClass object or other Container. And that all with possibility to import this ODB into a ZServer ZODB and to see my objects over URL. Thanks, Petr -- Petr Knápek NEXTRA Czech Republic, s.r.o., Veveří 102, 659 10 Brno, Czech Republic e-mail: mailto:petr.knapek@nextra.cz tel:+420-5-41 558 394 FAX:+420-5-41 558 390 "Perl is worse than Python because people wanted it worse." Larry Wall, 14 Oct 1998

1 0

Memory Management How-To
by Jason Spisak 31 May '00

31 May '00

Zopists, Can anyone see a document before it's been Cataloged? If so, check out: http://www.zope.org/Members/jspisak/memory and help me fill in the part where I just don't know enough. If not, then I'll request a review. -- Jason Spisak 444(a)hiretechs.com

1 0

RE: Memory issue 2.1.1 wrap up (Long)
by Jason Spisak 31 May '00

31 May '00

Thanks to everyone who gave me advise on what was going on. I think the problem was that I need some tuning in a variety of areas. (One of them being my brain) I am going to write up a little ditty on what I discovered so that people can check it before they claim leakage is occuring. I don't know enough about OS's to really put the *technical* explanations of some of the things to look for so, Matt, if you'd take a look at it when I post the url, I'd appreciate it. Just a quick rehash-- 1. I was running on an SMP machine, andthe setcheckinterval in Python wasn't helping me realize the machine's potential. This contributed to the slow down. Points:(2 physical, 1 psychological) 2. The 'Cache' tab in Zope was set to give objects a long, leisurely time in memory. This was causing Zope to rapidly claim and hold onto a ton of memory for objects it wan't even using. I adjusted it to 3 seconds. I worked the number until I found a happy medium. (BTW, I didn't really know *exactly* what this meant, and I hope there's a help system file on it in 2.2) Points:(1 physical, 1 psychological) 3. The 'Cache tab in Zope was set to allow a large amount of objects (20,000) to be a target size. This made those large amounts of objects that were lying around just happy to be there, I guess, I'm stil not sure what it does exaclty. Points:(1 physical, 1 psychological) 4. I store ton of information in my ZCatalog. The way the ZCatalog does an update (from what Shane told me) is a memory intensive task, loading lots of objects (I'd love help here), and when you combine that with 2 & 3, you get maxed out pretty quick. Catalog updates were taking a long time. Points:(2 physical, 2 psychological) 5. I use the standard Folder object to store 12,000 objects. Whenever a new object is added there is a huge hit on performance. Also, that object just so happens to get ZCataloged! That's turns Zope into a dog for a bit. [A good solution here is ZPatterns' RackMountable, but this is a production environment, and I want to let ZPatterns stabilize a bit. I am very excited about it though.;)] Points:(2 physical, 2 psychological) 6. The 'free' command on GNU/Linux shows just how much Linux buffers. A ton. Linux basically took my box from 1 gig of memory down to 1 MB, everytime. And that's is where it likes to stay. That's scary, but not fatal. Then the swap increased to about 40MB. When staying right on top of those numbers, it looks pretty bad while it's happening, so I believed things were headed down the drain. Zope's performance was poor (due to 1,2,3,4,5), so I thought the end was near. "If I don't restart Zope, things will only get worse." This is not true, the swap has hovered at 12MB average, and the free mem is averaging 4MB bobbing like a rubber duck. Points:(1 physical, 4 psychological) 7. There is in fact a memory leak in the SQLSession product. ;) Points:(0 physical, 2 psychological) All of these together with the fact that I learn more everyday about Zope and Linux, and you have the prime situation for Memorious Falsis. A syndrome where the hacker sees memory leaks that aren't there. Thanks wholeheartedly, -- Jason Spisak 444(a)hiretechs.com

1 0

RE: [Zope-dev] 2.2.0a1 won't work with query objects
by T.J. Mannos 31 May '00

31 May '00

> I think this is actually Zope's bug - if I'm not mistaken, the > TinyTables product uses the Results class provided in > lib/python/Shared/DC/ZRDB/Results.py, which would make it > subject to breaking in the same way that SQLMethod results > broke in the 2.2 alpha. I've included a copy of the note I > sent to the folks reporting problems with SQLMethods that > includes the patch you can use to verify that it fixes > the problem for TinyTables as well: That got me further, but it's still not all the way there: Traceback (innermost last): File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 224, in publish_module File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 189, in publish File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 175, in publish File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/mapply.py, line 160, in mapply (Object: index.html) File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 112, in call_object (Object: index.html) File /usr/Zope-2.2.0a1-src/lib/python/OFS/DTMLDocument.py, line 168, in __call__ (Object: index.html) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_String.py, line 500, in __call__ (Object: index.html) File /usr/Zope-2.2.0a1-src/lib/python/OFS/DTMLMethod.py, line 156, in __call__ (Object: standard_html_header) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_String.py, line 500, in __call__ (Object: standard_html_header) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_Util.py, line 333, in eval (Object: REQUEST.set('login',validateUser(_.None, _))) (Info: REQUEST) File <string>, line 0, in ? File /usr/Zope-2.2.0a1-src/lib/python/OFS/DTMLMethod.py, line 156, in __call__ (Object: validateUser) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_String.py, line 500, in __call__ (Object: validateUser) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_With.py, line 146, in render (Object: PARENTS[0].sessions) File /usr/Zope-2.2.0a1-src/lib/python/OFS/DTMLMethod.py, line 156, in __call__ (Object: getLogin) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_String.py, line 500, in __call__ (Object: getLogin) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_Util.py, line 333, in eval (Object: SESSION.has_key('login')) (Info: SESSION) File <string>, line 0, in ? File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_Util.py, line 140, in careful_getattr File /usr/Zope-2.2.0a1-src/lib/python/OFS/DTMLMethod.py, line 180, in validate (Object: index.html) File /usr/Zope-2.2.0a1-src/lib/python/AccessControl/SecurityManager.py, line 139, in validate File /usr/Zope-2.2.0a1-src/lib/python/AccessControl/ZopeSecurityPolicy.py, line 160, in validate Unauthorized: has_key So close! Many of my queries work, but a few break when trying to access a simple property or value (such as has_key). Some even break inconsistently, such as my getInstructorInfo query, which works fine inside a DTML document but doesn't work at all inside a Z SQL method. Here's its traceback: You are not authorized to access getInstructorInfo. Traceback (innermost last): File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 224, in publish_module File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 189, in publish File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 175, in publish File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/mapply.py, line 160, in mapply (Object: edit_instructors) File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 112, in call_object (Object: edit_instructors) File /usr/Zope-2.2.0a1-src/lib/python/OFS/DTMLDocument.py, line 168, in __call__ (Object: edit_instructors) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_String.py, line 500, in __call__ (Object: edit_instructors) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_With.py, line 146, in render (Object: REQUEST) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_Util.py, line 333, in eval (Object: db.updateInstructorInfo(login=login, name=name, email=email, phone=phone, office=office, homepage=homepage, password=password)) (Info: phone) File <string>, line 0, in ? File /usr/Zope-2.2.0a1-src/lib/python/Shared/DC/ZRDB/DA.py, line 432, in __call__ (Object: updateInstructorInfo) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_String.py, line 500, in __call__ (Object: <string>) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_Util.py, line 327, in eval (Object: getInstructorInfo(login=login)) (Info: getInstructorInfo) Unauthorized: (see above)

1 0