October 2006 - Zope - Zope lists

REQUEST Variables and Caching
by Dyyryath 20 Oct '06

20 Oct '06

Can anyone here give me a quick explanation of how the 'REQUEST variables' field works in the RAM Cache Manager properties? I've looked and all I can find is: ---------------------------------------- This information is used to store the cached copy of a page. This is an advanced feature, for now, you can leave this set to just "AUTHENTICATED_USER". ---------------------------------------- I'm trying to ensure that the cache manager only caches things for users who are NOT logged in.

2 1

not allowed to access a particular tuple
by William Heymann 20 Oct '06

20 Oct '06

This is an external method I wrote to make it easier to iterate over the records returned from searching the catalog. However if I have yield record,item instead of yield item it says that I am not allowed to access a particular tuple during iteration. How can I make it so that the value that it yields is something that a python script object can iterate over? I have also tried doing yield tuple((record,item)) yield list(record,item) and they give the same problem. How I use the script is pretty much for record, doc in catalogIter(context.Catalog(searchterms)): print record print doc Of course what is inside is different from that but it will cause the same problem since the issue happens in the iteration not inside the for loop. import zExceptions def catalogIter(records): "iterate over a catalog safely" for record in records: try: item = record.getObject() if item is not None: yield item except (zExceptions.Unauthorized, zExceptions.NotFound, KeyError, AttributeError): pass

3 3

Resolving a url from a path in a virtual host environment
by Pablo Ziliani 20 Oct '06

20 Oct '06

Hi guys, I'm having some trouble tying to get an object from a given path. I'm using virtual hosts, so according to Dieter's great documentation[1], REQUEST.resolve_url should do it. However no matter how creative I get making up urls, I always get a "Different namespace." ValueError. Looking at the source of resolve_url[2] I can see that it tries to ensure being at the same namespace (?) seeing if REQUEST.script is at the beginning of the passed url. So the actual problem is that this attribute always returns "http://127.0.0.1:8080" in my (2.9.4-final) Zope instance, quite independently from my requested url. So: should I be using some other method instead? (if so, I guess my reference is wrong or outdated). Which one? Do I need any special configuration to make REQUEST.script returns my virtual environment? Any additional suggestion? In case the real url after apache's rewrite is relevant, PATH_INFO and PATH_TRANSLATED are showing something like: '/VirtualHostBase/http/www.mydomain.com:80/sites/mysite/VirtualHostRoot/foo/bar.py' TIA, Pablo [1] : Quoting http://www.dieter.handshake.de/pyprojects/zope/book/chap3.html#c37ac15c11b4: "Unless your site uses /virtual hosts/ (or the virtual hosting tools for a different purpose) restrictedTraverse is almost the inverse of absolute_url. [...] The true inverse of absolute_url is the method resolve_url of the request object." [2] http://svn.zope.org/Zope/trunk/lib/python/ZPublisher/HTTPRequest.py?rev=687…

4 4

Zope 2.9 vs Zope 2.10 and Unicode problems
by Doyon, Jean-Francois 20 Oct '06

20 Oct '06

Hello, I just upgraded from Zope 2.9.4 to Zope 2.10 and am running into the common type of error: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 295: ordinal not in range(128) I've tracked it down to the fact that in Zope 2.10 it looks like the result of tag:replace tags (like the img/tag TAL), returns a unicode string: u'<img src="http://<host>/jf293/canada_wordmark.gif" alt="" title="" height="21" width="86" />' (In Zope 2.9.4 this was a normal string) This creates a problem because the PT is in ISO-8859-1 and contains french accents in non unicode strings, such as in the <meta> tags. <meta name="dc.creator" content="Gouvernement du Canada, Ressources Naturelles Canada, Secteur des sciences de la terre, Géomatique Canada - Centre canadien de télédétection, Division GéoAccès, Section de l'infrastructure de l'Atlas" /> Which looks like this by the time StringIO.getvalue() sees it: <meta name="dc.creator" content="Gouvernement du Canada, Ressources Naturelles Canada, Secteur des sciences de la terre, G\xe9omatique Canada - Centre canadien de t\xe9l\xe9d\xe9tection, Division G\xe9oAcc\xe8s, Section de l\'infrastructure de l\'Atlas" /> So I now end up with the UnicodeDecodeError :( How would I got about making sure that either my HTML ends up in unicode strings, or making sure the tal:replace (and possibly others?) don't use unicode strings? Both these things are probably rooted deep in the zope 2 or zope 3 code? My locale is set to en_CA and the default charset is iso-8859-1 ... Any ideas? Thanks in advance! Jean-François Doyon Internet Service Development and Systems Support / Spécialiste de dèveloppements internet et soutien technique Canada Centre for Remote Sensing/Centre Canadien de télédétection Natural Resources Canada/Ressources Naturelles Canada http://atlas.gc.ca Tel./Tél.: (613) 992-4902 Fax: (613) 947-2410

2 2

How to install POPMAIL for Zope 2.9?
by Jaroslav Lukesh 20 Oct '06

20 Oct '06

Hi Zopistas, I hace code which uses pop library from this product: http://www.zope.org/Members/dshaw/POPMailProduct/install Whet I do Import from Products page (see install), then I get message: The object POPMail does not support this operation. Did someone know how is possible way to install this product at Zope 2.9, please,, Many thanks, J. Lukesh

2 1

MailBoxer
by Daniel Raible 20 Oct '06

20 Oct '06

is it possible to use Mailboxer to send emails to all registered users - if yes how is this being accomplished? is it a good/bad idea? are there better solutions? thanks -dr

2 1

external method strangeness
by garry saddington 19 Oct '06

19 Oct '06

Can anyone explain what is happening here. I am using Saxon to transform an xml file(source) using a stylesheet(style) the result is then written to Postgres. The contents of source are uploaded in Zope and passed to the external method as a string. When I run this external method: import psycopg, string, re, subprocess def scholarpack_xml_transform(source): f=open('/opt/scholarpack/ancillary/source.xml','w') f.write(source) f.close source= '/opt/scholarpack/ancillary/source.xml' style='/opt/scholarpack/ancillary/style.xml' source1=source.replace(''','`') source2=source1.replace('NLPGaddress','BS7666address') p=re.compile("\'") source3=p.sub( '`' , source2) r = subprocess.Popen(['/opt/scholarpack/ancillary/jre/bin/java','-jar','./saxon.jar',source3,style], stdout = subprocess.PIPE,cwd = '/opt/scholarpack/ancillary/') r.wait() transformed = r.stdout.read() transformed1=transformed.replace(',)',')') conn = psycopg.connect("dbname=scholarpack user=scholarpack") curs = conn.cursor() do=curs.execute(transformed1) do conn.commit() conn.close I get a programming error thrown from the conn.commit() line and the method fails. However when I run the following script externally to Zope after the Zope failure (so that source.xml exists) the DB write is successful. #!/opt/scholarpack/python/bin/python import psycopg, string, re, subprocess source='source.xml' style='style.xml' source1=source.replace(''','`') source2=source1.replace('NLPGaddress','BS7666address') p=re.compile("\'") source3=p.sub( '`' , source2) r = subprocess.Popen(['/opt/scholarpack/ancillary/jre/bin/java','-jar','./saxon.jar',source3,style], stdout = subprocess.PIPE,cwd = '/opt/scholarpack/ancillary/') r.wait() transformed = r.stdout.read() transformed1=transformed.replace(',)',')') conn = psycopg.connect("dbname=scholarpack user=scholarpack") curs = conn.cursor() do=curs.execute(transformed1) do conn.commit() conn.close Can anyone point me in the correct direction? regards garry

4 6

Zope and Log File Analysis
by Doyon, Jean-Francois 19 Oct '06

19 Oct '06

Hello, So I've been struggling for years now to try and provide thorough log analysis for my Zope based web site. The big problem I have is determining "file types" without having "file extensions", since many objects are name with a simple id that doesn't contain a typical file extension. This makes it supremely difficult to determine viewing statistics based on content type for example ... Or determining what should be concidered a "page view", and so on. The best solution I can think of is to actually come up with standard file extensions for the various content types I use ... It's slightly annoying because of the "dot" in the ID, but otherwise, it should work ok. Has anybody confronted this issue before? Any other alternatives I haven't concidered? Any insight would be much appreciated! Thanks, Jean-François Doyon Internet Service Development and Systems Support / Spécialiste de dèveloppements internet et soutien technique Canada Centre for Remote Sensing/Centre Canadien de télédétection Natural Resources Canada/Ressources Naturelles Canada http://atlas.gc.ca Tel./Tél.: (613) 992-4902 Fax: (613) 947-2410

4 7

Re: zope fork in external method - mysql connection dropped
by Daniel Lopez 18 Oct '06

18 Oct '06

I made one tweak to the double-fork procedure, adding a waitpid call in the grandparent process (the original zope thread) before it returns out of the external method... the code then looked something like: [...prefork code up to here...] pid1 = os.fork() if pid1 > 0: #grandparent waits for its child before returning os.waitpid(pid1, 0) return RESPONSE.redirect("wait_page") pid2 = os.fork() os.setsid() if pid2 > 0: #child quits, orphaning grandchild sys.exit(0) [...grandchild-only code after here...] The waitpid call seems to be preventing the zombies... this is good! (though if you find something bad about this approach, please do speak up) But in the process, a new bug was created, having to do with the MySQL connection. I now receive a "Lost connection to MySQL server during query" error in what appears to be a final db flush from the grandparent's publish function (ZPublisher.Publish, line 104). The good news is that the grandchild continues to do it's work, but the bad news is that the user receives an error page instead of the redirect to the "wait_page". My guess is that this has something to do with the various forked processes sharing the MySQL connection, and in particular, I'm guessing the child process that exit(0)'s is closing the connection, which is then unavailable when the grandparent tries to use it. Here's the interesting part: if I put a sleep(1) line in between the waitpid() line and the return line, I get no error. It would appear that the 1 second pause provides enough time for a new MySQL connection to be made after the previous one is killed (less than 1 second is not enough). This is a very ugly fix, though... I can't guarantee that 1 second will always be enough... can anyone suggest a cleaner solution? Perhaps I shouldn't kill the child with sys.exit(0)? -Daniel On Fri, 13 Oct 2006, Jonathan wrote: > Pls keep your posts on the list - so that others can help and so that others > can search for problems/solutions! > > ----- Original Message ----- > Sent: Friday, October 13, 2006 12:13 PM > Subject: zope fork in external method > > >> >> Jonathan- >> >> I'm emailing you because I saw your post a few months back at: >> >> http://mail.zope.org/pipermail/zope/2006-May/166574.html >> >> I've been doing my best to find answers on existing posts, but to no avail. >> Perhaps you can send further pointers? >> >> On my system, a user can hit a page that initiates a long database query. If >> the user hits the stop button and tries to refresh, it messes things up (by >> interrupting the code upon return of the query), so I came up with a >> solution that almost works: >> >> I put all the code into an external method, which forks. The parent returns >> a redirect to a "waiting" throbber page that uses Ajax to poll whether the >> query is done. The child then goes on to do the hard work. >> >> This is working as desired, with a single exception: zombie child processes. >> Where can I learn more about forking processes in zope external methods? >> I've read about the double-fork method, but that hasn't resolved the >> problem... > > A few ideas for reaping dead child processes: > > 1) implement a SIGCHLD handler > > 2) when a spawned child process is finished its 'zope processing' have it > write its process id to a file (be careful when accessing files, you will > need locking to eliminate problems that may be caused by zope's multiple > threads) and then have a clean up routine which kills all of the processes > listed in the file (this could be an independant clean-up routine which wakes > up on a regular basis, or it could be built into the 'spawning' process which > does the clean up before it spawns a child). > > 3) as the last thing it does, have the child process issue a kill -9 system > command to kill its own process (I haven't tried this myself, but it may work > and it would be simple to implement) > > Good luck! > > Jonathan > > >

4 5

Tab completion in zopectl debug
by Maurits van Rees 18 Oct '06

18 Oct '06

Hi, I have recently learned how to add tab completion to an interactive python session and to pdb, thanks to Tom Lazar and to Jacob at ParTecs. More info with links on my first blog entry in three months: http://maurits.vanrees.org/weblog/2006/10/tab-completion.html At the end of that entry I ask a question: Can we add tab completion to the zopectl debugger? Or actually more general: can we load the ~/.pythonrc.py file when the zopectl debugger is started? In that file, the user can arrange tab completion or anything else he wants to have available for interactive python sessions. It is very easy to do: just add 'import user;' to the command line in the do_debug function of $SOFTWARE_HOME/Zope2/Startup/zopectl.py. For details, see that blog entry. I don't think there are security risks (but don't take my word for it). If someone can put malicious python code in that rc file, than that someone could just as easily start a python session and enter the malicious code there. This would just be for convenience. Would this be enough of a convenience to put into the Zope source code? -- Maurits van Rees | http://maurits.vanrees.org/ [NL] Work | http://zestsoftware.nl/ "Do not worry about your difficulties in computers, I can assure you mine are still greater."

6 8