May 2002 - Zope-Dev - Zope lists

debugging wedged zopes
by Leonardo Rochael Almeida 09 May '02

09 May '02

Hi guys, You guys remember that Zope of mine that used to segfault? well, it obviously isn't segfaulting anymore, but now it's behaving in a weird manner. I'm positive it has nothing to do with the segfaults it used to have (don't worry, Matt, I won't drag you to IRC once again :-), but to the complex Zope application it is running (it's full of ZClasses and PythonsScripts and stuff). sometimes it's sitting there nicely by itself at 196MB of RSS memory, suddenly it starts eating memory like a pig and responding VERY slowly. When that happens, usually one of Zope threads is pegged at ninety-something %CPU. Is there a way to know what a Zope thread is doing at a certain point in time, or at least what request it is trying to serve? If I where to roll my own, where would I get the data to do a data structure to keep track of what request the threads are servicing? Cheers, Leo -- Ideas don't stay in some minds very long because they don't like solitary confinement.

4 4

passing request to product methods
by Alexander Westholm 09 May '02

09 May '02

All, I'm passing form data to a method in a product through the REQUEST variable, i.e. <dtml-var expr="MyMethod(REQUEST)"> ...there has to be a way to do this implicitly so that users of the product can just call <dtml-var MyMethod> Any help is much appreciated. -- Alex

2 1

RAMCachemanager not caching brains or why doesn't Zope like the taste of pickled brains ?
by Tim Hoffman 09 May '02

09 May '02

Hmm I shall investigate why <mybrains> instances can't be pickled as it strikes me that they are probably one of the more likely objects one would like to cache (despite their taste ;-) ie results of expensive catalog searches that will be batched. === time elapsed ===== And now I know why I can't pickle brains in Products.Catalog.Catalog The following method declare the class mybrains inside it. def useBrains(self, brains): """ Sets up the Catalog to return an object (ala ZTables) that is created on the fly from the tuple stored in the self.data Btree. """ class mybrains(AbstractCatalogBrain, brains): pass scopy = self.schema.copy() scopy['data_record_id_']=len(self.schema.keys()) scopy['data_record_score_']=len(self.schema.keys())+1 scopy['data_record_normalized_score_']=len(self.schema.keys())+2 mybrains.__record_schema__ = scopy self._v_brains = brains self._v_result_class = mybrains And looking at the doc's seb pointed too, it says that picklable classes must be top level classes. So two questions are raised from this 1. has mybrains been declared within the method for a specific reason such as data hiding, and/or to prevent pickling it? 2. What would be the danger of caching brains ? Regards Tim On Wed, 2002-05-08 at 21:42, seb bacon wrote: > On Wed, 2002-05-08 at 05:36, Tim Hoffman wrote: > > Hmm, any ideas on why brains wouldn't be picklable > > *guessing* brains probably include non-top-level functions or classes: > > http://www.python.org/doc/current/lib/node62.html > > seb > > > > _______________________________________________ > Zope-Dev maillist - Zope-Dev(a)zope.org > http://lists.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://lists.zope.org/mailman/listinfo/zope-announce > http://lists.zope.org/mailman/listinfo/zope ) > > > > _______________________________________________ > Zope-CMF maillist - Zope-CMF(a)zope.org > http://lists.zope.org/mailman/listinfo/zope-cmf > > See http://www.zope.org/Products/PTK/Tracker for bug reports and feature requests

2 1

Re: [Zope-dev] DTML is not simple
by Ken Manheimer 08 May '02

08 May '02

(I missed the start of this topic, plz forgive me if i belabor what's already been said...) > Jim Penny wrote: > > > > But, I also agree with Adrian's thrust, DTML is a simple useful > > language, that, ignoring the quoting problems and all the _ namespace > > magic, can be picked up by anyone with an imperative language background > > in a couple of days. (Now, the quoting problems are indeed gross). > > There is also far too much magic in DTML, especially in name lookup. > > So you see, DTML isn't actually simple ;-) > > It appears simple, until you try and do anything useful with it. > Then, all of a sudden, you have to use the quoting, the _, and you have to try and make > sure you're <dtml-var some_variable>'s are coming from the right place, not calling > something when they don't mean to, etc, etc... As far as i'm concerned, DTML's ideosyncracies are insignificant compared with the fact that DTML tries to intermix procedural and layout/templating languages, which is fundamentally flawed. It can certainly be useful - like 'goto' can be useful in programming - but you lose crucial benefits of structured code in the process. You can still *program* with DTML - as you can still program with 'goto' - but you're dealing with a structuring mixture whose bizarreness doesn't help, and often hurts. The fact that you can close a DTML loop containing only part of an HTML element - eg, part of a table cell, or even part of an HTML tag - offers the kind of freedom whose cost is guarded against by principles of structured programming. ZPT, on the other hand, uses the template structuring as the code structuring - it reconciles the two. This constraining means that - The code can be checked for syntactic correctness - People familiar with just the layout aspects are not totally mislead, confused and thwarted by the added programming elements. - People working with the programming elements don't get thwarted by the layout people's (or their tools') unknowning disruption of the programming code. - Probably most important, in my view as a programmer, is that people working with the programming code can *adjust* what's there without going crazy trying to track down how it was supposed to fit with the layout code according to arbitrary, ad-hoc coupling. Like i said, i think this debate is very like the arbitrary-goto programming issue which was laid to rest years ago - see dijkstra's "Go To Statement Considered Harmful", from 1968, for a classic account (it's reprinted as an ACM "classic of the month", http://www.acm.org/classics/oct95/ ). (I'm fond of saying that ZPT does well the thing which makes zope excell - good impedence matching - and DTML does *not*.) -- Ken klm(a)zope.com

2 1

Threaded Database Access
by Jeff Nielsen / UgoFast 08 May '02

08 May '02

I'm using Zope/Zserver 2.2.4, ZODBCDA v3.1.0b2 and SQL Server 2000. I've noticed that pages generated from database querries are definitely single threaded, and these pages are the bulk of my site. My poking around seems to indicate that ZODBCDA is the single-threaded culprit, and some emails from the list over the last couple months seem to confirm that. The volume of my site and the size of the database (hence query response times) are getting to the point that this is a concern. So I'm investigating my options, including: 1) Cluster the web site with multiple single processor machines. 2) Move to a database that Zope supports multi-threaded access to. 3) ? I've been considering a move to Oracle or PostgreSQL/MySQL for other reasons, but I'm not going to do it just for yucks. Extending and populating my site gives better return than porting the database. I'd like to make sure I'll get return on my time investment. I saw an document on the Zope web site from 1999 that talked about labeling Zope DAs Level 1,2 or 3 based on their sophistication, but I don't see any of the current DAs labeled as such. There was text in the Oracle and MySQL DA descriptions that seem to imply multi-threaded behaivor though. I'm looking for the info from the community on: 1. Is the ZODBCDA single-threaded and might that change anytime soon? 2. Can someone confirm that the Oracle, PostgreSQL or MySQL DAs are multi-threaded. 3. Does the multi-threading require using the Zope Session facility or some other feature? (My site uses FSSession.) Thanks in advance for any input. Jeff Nielsen / UgoFast http://www.UgoFast.com Jeff(a)UgoFast.com

4 4

[PATCH] to Zope's Management Interface
by Jerome Alet 08 May '02

08 May '02

Hi the attached patch to apply to zope/lib/python/OFS/dtml/main.dtml allows people who want to use ZShell to type ZShell commands directly from the Zope's Management Interface. It should be transparent for people who don't want to use ZShell at all. to use this patch, you have to : - download ZShell v1.42 from : http://www.librelogiciel.com/software/ This is mandatory to stay in the ZMI when running commands, otherwise, with older ZShell versions you'll enter ZShell user interface after entering a command. - extract it in your home directory - cd to the Zope's home, e.g. /usr/lib/zope - type the following command (as root) : patch -p0 <~myusername/zshell-1.42/main.dtml.patch - restart Zope Enjoy !!! (At least you should) This was tested against Zope 2.5.0-4 (Debian package) but applying the patch manually to other versions if needed is very easy. As always any comment is very welcome. PS : This procedure and the patch are now included in the ZShell 1.42 package. Jerome Alet --- lib/python/OFS/dtml/main.dtml.orig Mon Apr 29 12:18:12 2002 +++ lib/python/OFS/dtml/main.dtml Mon Apr 29 12:38:34 2002 @@ -60,6 +60,15 @@ </table> </dtml-if> +<dtml-if "_.hasattr(this(), 'zshell')"> + <form action="zshell" method="GET"> + <input type="text" name="zshellscript:string" width="60" /> + <input type="hidden" name="zshellredirect:string" value="&dtml-URL0;" /> +   + <input type="submit" name="zshellbutton" value="ZShell!" /> + </form> +</dtml-if> + <form action="&dtml-URL1;/" name="objectItems" method="post"> <dtml-if objectItems> <table width="100%" cellspacing="0" cellpadding="2" border="0">

6 13

python cookie page calling dilemma
by Mark A. Robinson 08 May '02

08 May '02

Hi there. New to this mailing list. Hope I am in the right area. Here is what I am trying to do: a user selects 1 item from a list in a dtml document form. This item then affects the rest of that web session for the user. Therefore I would like the dtml document form to call a python script, with the item the user selected. The python script will do some database selects based on the selected item. It will also create a cookie with the selected item so that the rest of the web session will be "aware" of what the user selected. Lastly, the script should send the user to another web page that has some sort of confirmation of what the user selected (print the cookie). It is all working so far except for the very last part. When I call the confirmation DTML document using something like "return context.dpage_confirmation()" I get an KeyError for the confirmation page. The cookie is being set. I have verified it by viewing the confirmation page by itself. It is just when the page is called from the python script. Any help greatly appreciated. Thanks!

2 1

Re: [Zope-CMF] Chnages to ram-cache manager and CMF ?
by Tim Hoffman 08 May '02

08 May '02

Hmm OK I know understand why it is failing (weel up to a point) previously I had always cached the results of scripts that returned lists of simple strings. I am now returning a sorted/filtered list of brains and the following code in RAMCacheManager def __init__(self, index, data, view_name): try: # This is a protective barrier that hopefully prevents # us from caching something that might result in memory # leaks. It's also convenient for determining the # approximate memory usage of the cache entry. self.size = len(dumps(index)) + len(dumps(data)) except: raise CacheException('The data for the cache is not pickleable.') is barfing because dumps is raising an exception when trying to pickle a list of brains. This is unfortunate, as caching the page that uses this list doesn't help because I am only showing a small set of the list on each page. Hmm, any ideas on why brains wouldn't be picklable, or for that matter why it would be unsafe to cache brains as the cache manager is just using the dumps operation as a saftey net, it doesn't actually do anything with the pickle. Rgds Tim On Wed, 2002-05-08 at 11:48, Tim Hoffman wrote: > Hi > > I am running a CVS snapshot of CMF as of 19/4/2002. > > I am finding that the results of python scripts don't seem > to being cached by the ram cache manager, > ie they are associated, the are seen in the statistics page > but no results are ever cached, ie all columns have a value of 0. > > I can successfully cache the pagetemplate that calls the script > > I have noticed there is now a CMF Caching policy tool which seems > to have something to do with skin methods, any likely hood > this could have an affect > > previous use of this approach with CMF1.2 worked fine. > > Any thoughts, hints > > Rgds > > Tim > > > > > > _______________________________________________ > Zope-CMF maillist - Zope-CMF(a)zope.org > http://lists.zope.org/mailman/listinfo/zope-cmf > > See http://www.zope.org/Products/PTK/Tracker for bug reports and feature requests

3 2

Selecting an option from a drop-down list at run-time with DTML
by Richard McKinley 08 May '02

08 May '02

Hi, I have a drop-down list which is dynamically populated from a database lookup table using the <dtml-in> structure to iterate through the list of rows and add an <OPTION> to the drop-down for each row in the table. like so: <select name="Purpose"> <dtml-in sqlLookupResearchPurpose> <option value="<dtml-var "RESEARCH_PURPOSE">"> <dtml- var "RESEARCH_PURPOSE"> </option> </dtml-in> </select> I want to be able to add the SELECTED HTML attribute to a specific option which is determined only at runtime. Basically if the current value of research_purpose field equals the value of the purpose variable I want to insert the select attribute at this point. <dtml-if expr="<dtml-var "RESEARCH_PURPOSE"> == <dtml-var "PURPOSE">"> SELECTED </dtml-if> However, I get a syntax error. How do you usually test the condition when 2 variable are equal? Putting the code together: <select name="Purpose"> <dtml-in sqlLookupResearchPurpose> <option value="<dtml-var "RESEARCH_PURPOSE">" <dtml−if expr="<dtml-var "RESEARCH_PURPOSE"> == <dtml-var "PURPOSE">"> SELECTED </dtml−if> > <dtml-var "RESEARCH_PURPOSE"> </option> </dtml-in> </select> If anyone could show me how this is normally done or point me in the direction of any tutorials/how-tos etc. it would be appreciated.. Many thanks, Richard _________________________________________________________________ Send and receive Hotmail on your mobile device: http://mobile.msn.com

1 0

What appears to be a bug in RAMCacheManager which would seem to affect it's performance
by Tim Hoffman 08 May '02

08 May '02

Hi Can someone confirm my belief that the following code in Products/StandardCacheManagers/RAMCacheManager.py is incorrect (upon which I will raise a collector issue) try: from cStringIO import dumps except: from pickle import dumps I believe it should be try: from cPickle import dumps The outcome of this error is that the results from every object that is associated with a ram cachemanager instance is pickled to see if it can be cached. And by not using the optimised version, which will always happen because "dumps" is part of cPickle not cStringIO, we always get the "slow" version. Thanks Tim P.S. Doesn't pickling the cacheable results seem to be a very expensive way to determine the suitability of caching the object?

1 0