- Zope-Dev - Zope lists

ZODB article draft
by Andrew M. Kuchling 31 May '00

31 May '00

I've just put up the first draft of a little introductory article about ZODB and ZEO. Please check it out and offer comments. Comments from inexperienced people are particularly welcome; is it a decent introduction? Which bits are confusing? http://starship.python.net/crew/amk/python/writing/zodb-zeo.html BTW, I'm looking for a good place to send this article -- byte.com? Oreillynet/python? -- please let me know your suggestions through private e-mail. --amk

1 0

Traversal Stuff
by Chris Withers 31 May '00

31 May '00

Hi, I just remembered, something I really miss from my Mason days are dhandlers. What this offers is having a URL like: http://mysite.com/folder/object/some/parameters processed by the 'object' object with a parameter of '/some/parameters' or better still a 'URL objects' list as a parameter. This sounds like Traversal Interface stuff so I thought now would be a good time to mention it given the traversal interface is being re-done. What do people think? cheers, Chris PS: I'm off to dump this into the interfaces Wiki right now... ;-) PPS: I know ZSQL methods can do something similar so it must be possible ...

2 1

Re: [Zope-dev] Zope 2.2.0 alpha 1 released...
by Anthony Baxter 31 May '00

31 May '00

>>> Brian Lloyd wrote > To me, the more hairy issue is what if Joe *does* exist in the > different Zope installation, and you *do* want Joe to continue > to have ownership? Currently, you must either arrange for Joe > to do the import (which will give him ownership directly), or > import it and use an external method to assign ownership (which > is a pain). One thing we've thought of is that perhaps superuser > (and only superuser) could be able to assign ownership through > a web interface, which could make this sort of thing a bit > easier. Hm. How about some sort of toggle that allows an import to maintain ownership from the import file? Course, you need to specify who is _allowed_ to set that toggle, or else anyone who can import can set up any ownership that they like. Anthony -- Anthony Baxter <anthony(a)interlink.com.au> It's never too late to have a happy childhood.

6 6

Re: [zope-xml] XMLDocument beta release 3
by Uche Ogbuji 31 May '00

31 May '00

> This seems still to be rather linux-centric. Now with the 4DOM binaries for > windows released, would there be a way to use this on win32 without jumping > to a string of hoops? Now that we have "official" Windows Binaries for all the 4Suite tools I think the greatest barrier is dhaving to patch Zope. This will likely be addressed by providing a CVS branch for the XML integration and incorporating it all into the existing cross-platform compatible structure. Hopefully we'll have this sorted out very quickly, but at least we have eliminated 75% of the hoops in question. We're working on the rest. Thanks. -- Uche Ogbuji Principal Consultant uche.ogbuji(a)fourthought.com +01 303 583 9900 x 101 Fourthought, Inc. http://Fourthought.com 4735 East Walnut St, Ste. C, Boulder, CO 80301-2537, USA Software-engineering, knowledge-management, XML, CORBA, Linux, Python

1 0

Re: [Zope-dev] Memory 2.1.4-2.1.6 a.k.a. how to get objects out of the cache
by Jason Spisak 31 May '00

31 May '00

Eric Sattler: > Jason, > > I don't have time right now to post this to the zope > newsgroup, but we are bleeding profusely, and could > use any insight you might have... > Unfortunately just before recieving your message, I fired another one out because I have run out of tests. :( > > I also am having severe problems with memory creep. Our problem > exhibits itself by quickly using gobs of memory and eventually requiring > a > zope restart. > > First our configuration: > > Core Components: > > FreeBSD 4.0/FreeBSD 3.4 - on separate machines of course > Postgres 6.5.3/Postgres 7.0(fixes some memory leaks) > Python 1.5.2 > Zope 2.1.4/2.1.6 > Apache w/mod_ssl 1.3.9 and 1.3.12 > > Products: > > ZPySQLDA > SQLSession > GenericUserFolder > There was recently a post that detained a leak in a database product. I can't remeber if it was SQLSession, or what. > I have looked at a lot of different things. This includes: > > Installing the gc(garbage collection) python package > Installing Postgres 7.0(fixes some connection memory leaks) > Tracking the cache cleanup inside zope > Tracking thread locking inside DB.py, Transaction.py > > Results: > > gc python didn't help alleviate the problem > Postgres 7.0 instead of 6.5.3 didn't help the problem > > I do see GenericUserFolder and SQLSession objects with the > Control_Panel_Debug screen, and they do not seem to go away. I wrote > a simple python script to do nothing more than authenticate (log in) > using the GenericUserFolder method docLogin. The memory usage quickly > grows out of control. After waiting 15 minutes(my cookie timeout), > no decrease in memory usage. The objects are still in the cache also. This is consistant with what I remember seeing on the list. I don't use any external databases, so my situation is really an argument for their being something amiss in Zope or Python. > > Questions: > > 1. How does a user's connection resources etc. get cleaned up > by Zope after a cookie timeout? Do I need to do this myself? > I don't use cookie here. > 2. How does a genericuserfolder's set of objects used by an > > authenticated user get cleaned up. I can't seem to make those > objects get reclaimed by the system, even when the user logs > off? is there a magic cache setting somewhere? > Or GUF. > 3. I am not explicitly removing SQLSession objects. Will my usage > counts for these objects remain > 1, thereby never allowing them > to get cleaned up, and thereby keeping my usage counts for > genericuserfolder's > 1? > > 4. Since GenericUserFolder inherits from Folder objects, is it > possible that the reason my memory grows so fast is that Zope > does not release the resources used by a user properly, during > an abrupt disconnect? That's funny that I am experienceing the same symptoms as you, but my installation is totally different. I'm not using GUF, or any DA's, and I'm running on RH 6.1. We really need to figure this out as a community. I am willing to try just about anything short of shipping my machine somewhere off premisis. ;) Bledding on the floor, Jason Spisak CIO HireTechs.com 6151 West Century Boulevard Suite 900 Los Angeles, CA 90045 P. 310.665.3444 F. 310.665.3544 Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.

3 4

Put Zope objects into a ZODB
by Petr Knapek 31 May '00

31 May '00

Hi all, I would like to use ZODB3 as an object DB to store my objects. I know How-To by Michael Pelletier on this topic. So that is to store objects in ZODB out of Zope. I prefer to store objects which are subclassed of Zope objects (especially Folder and ZClass - not multiple) because in future I would like to import this my ZODB into a Zope ZODB as its part into a folder. Could anybody help me where to find any information, documentation ... My task is to use ZODB as object database to storage objects. Containers, subclassed from Zope Folder which will store other objects sublassed from ZClass object or other Container. And that all with possibility to import this ODB into a ZServer ZODB and to see my objects over URL. Thanks, Petr -- Petr Knápek NEXTRA Czech Republic, s.r.o., Veveří 102, 659 10 Brno, Czech Republic e-mail: mailto:petr.knapek@nextra.cz tel:+420-5-41 558 394 FAX:+420-5-41 558 390 "Perl is worse than Python because people wanted it worse." Larry Wall, 14 Oct 1998

1 0

Memory Management How-To
by Jason Spisak 31 May '00

31 May '00

Zopists, Can anyone see a document before it's been Cataloged? If so, check out: http://www.zope.org/Members/jspisak/memory and help me fill in the part where I just don't know enough. If not, then I'll request a review. -- Jason Spisak 444(a)hiretechs.com

1 0

RE: Memory issue 2.1.1 wrap up (Long)
by Jason Spisak 31 May '00

31 May '00

Thanks to everyone who gave me advise on what was going on. I think the problem was that I need some tuning in a variety of areas. (One of them being my brain) I am going to write up a little ditty on what I discovered so that people can check it before they claim leakage is occuring. I don't know enough about OS's to really put the *technical* explanations of some of the things to look for so, Matt, if you'd take a look at it when I post the url, I'd appreciate it. Just a quick rehash-- 1. I was running on an SMP machine, andthe setcheckinterval in Python wasn't helping me realize the machine's potential. This contributed to the slow down. Points:(2 physical, 1 psychological) 2. The 'Cache' tab in Zope was set to give objects a long, leisurely time in memory. This was causing Zope to rapidly claim and hold onto a ton of memory for objects it wan't even using. I adjusted it to 3 seconds. I worked the number until I found a happy medium. (BTW, I didn't really know *exactly* what this meant, and I hope there's a help system file on it in 2.2) Points:(1 physical, 1 psychological) 3. The 'Cache tab in Zope was set to allow a large amount of objects (20,000) to be a target size. This made those large amounts of objects that were lying around just happy to be there, I guess, I'm stil not sure what it does exaclty. Points:(1 physical, 1 psychological) 4. I store ton of information in my ZCatalog. The way the ZCatalog does an update (from what Shane told me) is a memory intensive task, loading lots of objects (I'd love help here), and when you combine that with 2 & 3, you get maxed out pretty quick. Catalog updates were taking a long time. Points:(2 physical, 2 psychological) 5. I use the standard Folder object to store 12,000 objects. Whenever a new object is added there is a huge hit on performance. Also, that object just so happens to get ZCataloged! That's turns Zope into a dog for a bit. [A good solution here is ZPatterns' RackMountable, but this is a production environment, and I want to let ZPatterns stabilize a bit. I am very excited about it though.;)] Points:(2 physical, 2 psychological) 6. The 'free' command on GNU/Linux shows just how much Linux buffers. A ton. Linux basically took my box from 1 gig of memory down to 1 MB, everytime. And that's is where it likes to stay. That's scary, but not fatal. Then the swap increased to about 40MB. When staying right on top of those numbers, it looks pretty bad while it's happening, so I believed things were headed down the drain. Zope's performance was poor (due to 1,2,3,4,5), so I thought the end was near. "If I don't restart Zope, things will only get worse." This is not true, the swap has hovered at 12MB average, and the free mem is averaging 4MB bobbing like a rubber duck. Points:(1 physical, 4 psychological) 7. There is in fact a memory leak in the SQLSession product. ;) Points:(0 physical, 2 psychological) All of these together with the fact that I learn more everyday about Zope and Linux, and you have the prime situation for Memorious Falsis. A syndrome where the hacker sees memory leaks that aren't there. Thanks wholeheartedly, -- Jason Spisak 444(a)hiretechs.com

1 0

RE: [Zope-dev] 2.2.0a1 won't work with query objects
by T.J. Mannos 31 May '00

31 May '00

> I think this is actually Zope's bug - if I'm not mistaken, the > TinyTables product uses the Results class provided in > lib/python/Shared/DC/ZRDB/Results.py, which would make it > subject to breaking in the same way that SQLMethod results > broke in the 2.2 alpha. I've included a copy of the note I > sent to the folks reporting problems with SQLMethods that > includes the patch you can use to verify that it fixes > the problem for TinyTables as well: That got me further, but it's still not all the way there: Traceback (innermost last): File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 224, in publish_module File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 189, in publish File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 175, in publish File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/mapply.py, line 160, in mapply (Object: index.html) File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 112, in call_object (Object: index.html) File /usr/Zope-2.2.0a1-src/lib/python/OFS/DTMLDocument.py, line 168, in __call__ (Object: index.html) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_String.py, line 500, in __call__ (Object: index.html) File /usr/Zope-2.2.0a1-src/lib/python/OFS/DTMLMethod.py, line 156, in __call__ (Object: standard_html_header) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_String.py, line 500, in __call__ (Object: standard_html_header) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_Util.py, line 333, in eval (Object: REQUEST.set('login',validateUser(_.None, _))) (Info: REQUEST) File <string>, line 0, in ? File /usr/Zope-2.2.0a1-src/lib/python/OFS/DTMLMethod.py, line 156, in __call__ (Object: validateUser) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_String.py, line 500, in __call__ (Object: validateUser) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_With.py, line 146, in render (Object: PARENTS[0].sessions) File /usr/Zope-2.2.0a1-src/lib/python/OFS/DTMLMethod.py, line 156, in __call__ (Object: getLogin) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_String.py, line 500, in __call__ (Object: getLogin) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_Util.py, line 333, in eval (Object: SESSION.has_key('login')) (Info: SESSION) File <string>, line 0, in ? File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_Util.py, line 140, in careful_getattr File /usr/Zope-2.2.0a1-src/lib/python/OFS/DTMLMethod.py, line 180, in validate (Object: index.html) File /usr/Zope-2.2.0a1-src/lib/python/AccessControl/SecurityManager.py, line 139, in validate File /usr/Zope-2.2.0a1-src/lib/python/AccessControl/ZopeSecurityPolicy.py, line 160, in validate Unauthorized: has_key So close! Many of my queries work, but a few break when trying to access a simple property or value (such as has_key). Some even break inconsistently, such as my getInstructorInfo query, which works fine inside a DTML document but doesn't work at all inside a Z SQL method. Here's its traceback: You are not authorized to access getInstructorInfo. Traceback (innermost last): File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 224, in publish_module File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 189, in publish File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 175, in publish File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/mapply.py, line 160, in mapply (Object: edit_instructors) File /usr/Zope-2.2.0a1-src/lib/python/ZPublisher/Publish.py, line 112, in call_object (Object: edit_instructors) File /usr/Zope-2.2.0a1-src/lib/python/OFS/DTMLDocument.py, line 168, in __call__ (Object: edit_instructors) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_String.py, line 500, in __call__ (Object: edit_instructors) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_With.py, line 146, in render (Object: REQUEST) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_Util.py, line 333, in eval (Object: db.updateInstructorInfo(login=login, name=name, email=email, phone=phone, office=office, homepage=homepage, password=password)) (Info: phone) File <string>, line 0, in ? File /usr/Zope-2.2.0a1-src/lib/python/Shared/DC/ZRDB/DA.py, line 432, in __call__ (Object: updateInstructorInfo) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_String.py, line 500, in __call__ (Object: <string>) File /usr/Zope-2.2.0a1-src/lib/python/DocumentTemplate/DT_Util.py, line 327, in eval (Object: getInstructorInfo(login=login)) (Info: getInstructorInfo) Unauthorized: (see above)

1 0

XMLDocument beta release 3
by Chimezie Thomas-Ogbuji 31 May '00

31 May '00

See release notes for changes (installation process simplified w/ release of 4DOM/4XSLT/4XPATH): -- Chimezie Thomas-Ogbuji Consultant Fourthought Inc. (303) 583 9900 ext 104 cogbuji(a)fourthought.com XMLDocument beta is a collection of Zope products and python tools that integrate XML into Zope, above and beyond the current XMLDocument product. Like ZDOM, it is the result of a collaborative venture by Digital Creations and Fourthought Inc. XmlDocument This is a Zope product that wraps XML content for use in Zope. In many ways it behaves similarly to the original XMLDocument, except it only creates a single Zope object upon import or creation. XMLImporter This python module allows users to externally import XML content into the Zope database in a 'controlled' manner. The user has full range of control over import parameters. XSLT Method This is a Zope product that wraps an XSLT stylesheet for use in Zope. Like a DTML method, it can be invoked on a ClobXmlDocument instance and return the resulting XSLT transformation. Fourthought provides custom technological solutions based on open technologies. 4Suite is a powerful set of Python tools developed by Fourthought for Web-based applications in areas such as Intranets, E-Commerce and knowledge-management. Elements of 4Suite have been adopted as core XML technologies in Zope and in the Python XML distribution links: Fourthought: http://fourthought.com 4Suite: http://fourthought.com/4Suite/ DOM 2 Spec: http://www.w3.org/TR/2000/CR-DOM-Level-2-20000307/ Zope http://www.zope.org contact: 4Zope(a)fourthought.com Installing XMLDocument beta --------------------------- This document describes installing XMLDocument beta on Unix. Important Notes XMLDocument requires 4DOM which is covered by the ZDOM installation. Simply installing ZDOM should resolve this dependency XMLDocument also requires 4XPath and 4XSLT You can download the rpms from here: ftp://ftp.fourthought.com/pub/mirrors/python4linux/redhat/i386/4XPath-0.9.0… ftp://ftp.fourthought.com/pub/mirrors/python4linux/redhat/i386/4XSLT-0.9.0-… Installing Download these files from our FTP server (ftp.fourthought.com) /pub/4Zope/XMLDocument/XMLDocument-beta.tgz /pub/4Zope/XMLDocument/XMLImporter-beta.tgz Installing XMLDocument and XSLTMethod: decompress XMLDocument-beta.tgz to your Zope directory Installing Demonstration: Import the testfolder.zexp file in $Zopeinstalldir$/Extensions. Point your browser to where you imported testfolder in the zope heirarchy and follow the directions. NOTE: The demonstration cyclicaly creates and destroys a ZDOM heirarchy in testfolder/, so if you stop it at mid point it may break if you try to restart it (if it's broken run the reset DTML Method: which simply removes folder1, folder2, and ZDOMtester) Installing XMLImporter: decompress XMLImporter-beta.tgz to an arbitrary directory. make sure (Zope install directory)/lib/python is in your PYTHONPATH * read the python files in test_suite for examples on how to use XMLImporter Assuming you've satisfied the dependencies, you should be done! questions? email: 4Zope(a)fourthought.com RELEASE NOTES 5/30/2000 1) Solved __call__ problem, XSLTMethods can be invoked programatically 2) Added a default view for XMLDocuments that can return the XML as: syntax highlighted HTML raw XML plain text this can be specified by the output property 3) fixed the copy paste bug 4) added demonstration/example to package 5/16/2000 Moved ClobXMLDocument and XSLTMethod to seperate products changed ClobXMLDocument to XMLDocument _______________________ XMLDocument beta supports the import and manipulation of XML files in Zope through the use of XSLT. See (XPath: http://www.w3.org/TR/1999/REC-xpath-19991116, and XSLT: http://www.w3.org/TR/1999/REC-xslt-19991116) XMLImporter is currently used as a python based tool to input XML files as Zope Objects. The mapping functionality and XSLT transform are used to manipulate the incomming XML stream into a set of Zope Objects. The transform is applied to the incoming XML to allow manipulation and compression of the incomming XML. See http://216.164.72.7:7780/Wiki/Members/jim/ZDOM/InitialNotes under the heading optimization. The final set of objects created by XMLImporter is defined by the mapping dictionary where keys are the tagNames to be created, and the values are references to python classes. A current limitation of the system is that Zope must not be running during an import. Clob XML Document is a Zope Object that represents a complete XML document. Interfaces are provided to edit the content of the document, and modify how the document is displayed when viewed. XSLTMethod is a Zope Object that applies an XSLT transform to an object. An XSLTMethod can be accessed as any other method of an object. With the ZDOM pakage installed, the XSLTMethod can be applied to any Object in the Zope application. Special processing is currently restricted to Clob XML Documents. When an XSLTMethod is applied to a Clob XML Document the contents of the document are used as a source, not the object itself. INSTALLATION --------------- Please see INSTALL.txt for Installation and package dependencies TODO ___________ 1. Allow XMLImporter to be called through the HTML interface 2. Allow Exporting of Zope Objects to XML (expand current export functionality) 3. Allow XSLT special processing on other objects (through _getDom interface) 4. Allow external parameters into XSLTMethod

2 1