January 2005 - Zope - Zope lists

Re: Problem with external method locking threads
by David Swift 25 Jan '05

25 Jan '05

Please belay this problem - it actually appears to be a problem with my choice of regular expression. the 'fileLink' regexp in the script goes into never-neverland for some links that have query strings containing links. Once in this state, Python has to be killed to stop it. Who'da thought infinite recursion could take so long... David

1 0

Problem with external method locking threads
by David Swift 24 Jan '05

24 Jan '05

I am running an External method, to fix some hard links in our ZWiki back to relative links. On two of our three ZWikis, this script runs well. On the third ZWiki, which has about 530 pages, this script causes the Zope instance to stop responding to any requests, after it processes about 140 pages. I have tried deleting the pages it freezes on, but it still freezes at about the same place each time. After examing the lists, and running strace, I am certain that one of the Python/Zope threads refuses to respond to SIGRTMIN, and then somehow locks up all threads - however, this is deeper than my knowledge goes, and I am hoping someone on this list can point me towards solutions I have not yet tried. My environment is: 2.5Ghz Intel single processor 1GB RAM Fedora 1 kernel - Linux our.server.com 2.4.22-1.2199.nptl #1 Wed Aug 4 12:21:48 EDT 2004 i686 i686 i386 GNU/Linux Software RAID 1, using a 36GB partition for Data.fs. Directory instance/var is a symlink to this partition. Python 2.3.3 and Python 2.4 Zope 2.7.3 ZWiki ZWiki-0-37-0 glibc-2.3.4-3 Below, I have copied: The External Method script I am using A brief description of the various things I have tried A sample of the environment, from within zopectl A copy of the strace run I am getting during script execution I use the command 'setuidgid zope bin/zopectl' to run the Zope instance. Any help is greatly appreciated! David -------------------- The script looks like this: from StringIO import StringIO import zLOG import re def addpage(self,o): """ Given a file or image, convert links to LocalFS """ nid = o.getId() fileOnlyLink = re.compile(r'href=\"([- .\w]+\.\w+)\"') fileLink = re.compile(r'https://zwiki(?:.*?/)+?([- .%\w]+.\w+)"') otherLink = re.compile(r'https://zwiki(?:.*?/)+([- %\w]+)"') pageText = o.read() pageText = fileOnlyLink.sub(r'href="uploads/\1"', pageText) pageText = fileLink.sub(r'uploads/\1"', pageText) pageText = otherLink.sub(r'\1"', pageText) o.setText(pageText) zLOG.LOG('addpage', zLOG.WARNING, 'working on %s' % nid) o.reindex_object() -------------------- tried Python 2.4 and Zope 2.7.3-0 tried Python 2.3.3 and Zope 2.7.3-0 tried refreshing glibc, to version glibc-2.3.4-3 tried LD_ASSUME_KERNEL=2.4.1 in runzope and zopectl tried LD_ASSUME_KERNEL=2.2.5 in runzope and zopectl tried deleting data for pages it stops on tried THREAD_STACK_SIZE of 0x20000 when compiling Python tried compiling Python with ./configure --prefix=/usr/local/python-2.4 --with-threads --with-signal-module --with-libs='rt' tried compiling Python with ./configure --prefix=/usr/local/python-2.4 --with-threads --with-signal-module tried compiling Python with ./configure --prefix=/usr/local/python-2.4 --with-threads tried compiling Python with ./configure --prefix=/usr/local/python-2.3.3 --with-threads --with-signal-module --with-libs='rt' tried compiling Python with ./configure --prefix=/usr/local/python-2.3.3 --with-threads --with-signal-module tried compiling Python with ./configure --prefix=/usr/local/python-2.3.3 --with-threads -------------------- environment of zopectl zopectl> stop . . . . . . . . . . . daemon process stopped zopectl> shell bash: /root/.bashrc: Permission denied bash-2.05b$ set BASH=/bin/bash BASH_ENV=/root/.bashrc BASH_VERSINFO=([0]="2" [1]="05b" [2]="0" [3]="1" [4]="release" [5]="i386-redhat-linux-gnu") BASH_VERSION='2.05b.0(1)-release' CLIENT_HOME=/usr/local/zope/apmindsf/var COLUMNS=140 DIRSTACK=() EUID=88 GROUPS=() G_BROKEN_FILENAMES=1 HISTFILE=/root/.bash_history HISTFILESIZE=1000 HISTSIZE=1000 HOME=/root HOSTNAME=rope.apmindsf.com HOSTTYPE=i386 IFS=$' \t\n' INPUTRC=/etc/inputrc INSTANCE_HOME=/usr/local/zope/apmindsf LANG=en_US.UTF-8 LD_ASSUME_KERNEL=2.4.1 LESSOPEN='|/usr/bin/lesspipe.sh %s' LINES=61 LOGNAME=root MACHTYPE=i386-redhat-linux-gnu MAIL=/var/spool/mail/root MAILCHECK=60 OPTERR=1 OPTIND=1 OSTYPE=linux-gnu PATH=/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/X11R6/bin:/root/bin PPID=8735 PS1='\s-\v\$ ' PS2='> ' PS4='+ ' PWD=/usr/local/zope/apmindsf PYTHONPATH=/usr/local/zope/lib/python SHELL=/bin/bash SHELLOPTS=braceexpand:emacs:hashall:histexpand:history:interactive-comments:monitor SHLVL=3 SOFTWARE_HOME=/usr/local/zope/lib/python SSH_CLIENT='192.168.1.20 3660 22' SSH_CONNECTION='192.168.1.20 3660 192.168.1.72 22' SSH_TTY=/dev/pts/1 TERM=cygwin UID=88 USER=root USERNAME=root ZMANAGED=1 ZOPE_HOME=/usr/local/zope _=/bin/bash bash-2.05b$ -------------------- strace of script execution [root@server Python-2.4]# strace -s2000 -p 8738 Process 8738 attached - interrupt to quit select(9, [3 4 8], [], [3 4 8], {4, 460000}) = 0 (Timeout) select(9, [3 4 8], [], [3 4 8], {30, 0}) = 1 (in [3], left {12, 590000}) accept(3, {sa_family=AF_INET, sin_port=htons(3429), sin_addr=inet_addr("192.168.1.20")}, [16]) = 14 fcntl64(14, F_GETFL) = 0x2 (flags O_RDWR) fcntl64(14, F_SETFL, O_RDWR|O_NONBLOCK) = 0 getpeername(14, {sa_family=AF_INET, sin_port=htons(3429), sin_addr=inet_addr("192.168.1.20")}, [16]) = 0 gettimeofday({1106285562, 100163}, NULL) = 0 fcntl64(14, F_SETFD, FD_CLOEXEC) = 0 select(15, [3 4 8 14], [], [3 4 8 14], {30, 0}) = 1 (in [14], left {30, 0}) recv(14, "GET /am/convertLinks2local/ZScriptHTML_tryForm HTTP/1.1\r\nHost: our.server.com:8080\r\nUser-Agent: Mozilla/5.0 (Windows; U; Wi ndows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0\r\nAccept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/pla in;q=0.8,image/png,*/*;q=0.5\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip,deflate\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,* ;q=0.7\r\nKeep-Alive: 300\r\nConnection: keep-alive\r\nReferer: http://our.server.com:8080/am/convertLinks2local/ZPythonScriptHTML_editFo rm\r\nCookie: tree-s=\"eJzTyCkw5NLIKTDiClZ3hANXW3WuAmOuxEQ9AIOOB9Q\"\r\nAuthorization: Basic ZHN3aWZ0OjR0aGZsMDBy\r\n\r\n", 4096, 0) = 643 gettimeofday({1106285562, 102930}, NULL) = 0 kill(8744, SIGRTMIN) = 0 kill(8744, SIGRTMIN) = 0 rt_sigprocmask(SIG_SETMASK, NULL, [RTMIN], 8) = 0 rt_sigsuspend([] <unfinished ...> --- SIGRTMIN (Unknown signal 32) @ 0 (0) --- <... rt_sigsuspend resumed> ) = -1 EINTR (Interrupted system call) sigreturn() = ? (mask now [RTMIN]) select(15, [3 4 8 14], [], [3 4 8 14], {30, 0}) = 1 (in [8], left {29, 900000}) read(8, "x", 8192) = 1 select(15, [3 4 8 14], [14], [3 4 8 14], {30, 0}) = 1 (out [14], left {30, 0}) send(14, "HTTP/1.1 302 Moved Temporarily\r\nServer: Zope/(Zope 2.7.3-0, python 2.4.0, linux2) ZServer/1.1\r\nDate: Fri, 21 Jan 2005 05:32:42 GMT\r\nBobo-Exception-Line: 56\r\nContent-Length: 0\r\nBobo-Exception-Value: See the server error log for details\r\nBobo-Exception-File: D T_Raise.py\r\nBobo-Exception-Type: Redirect\r\nConnection: close\r\nLocation: http://our.server.com:8080/am/convertLinks2local\r\nContent -Type: text/html;charset=iso-8859-1\r\n\r\n", 429, 0) = 429 select(15, [3 4 8 14], [], [3 4 8 14], {30, 0}) = 1 (in [14], left {30, 0}) rt_sigprocmask(SIG_SETMASK, NULL, [RTMIN], 8) = 0 rt_sigsuspend([] <unfinished ...> --- SIGRTMIN (Unknown signal 32) @ 0 (0) --- <... rt_sigsuspend resumed> ) = -1 EINTR (Interrupted system call) sigreturn() = ? (mask now [RTMIN]) recv(14, "", 4096, 0) = 0 close(14) = 0 select(9, [3 4 8], [], [3 4 8], {30, 0}) = 1 (in [8], left {30, 0}) read(8, "x", 8192) = 1 select(9, [3 4 8], [], [3 4 8], {30, 0}) = 1 (in [3], left {29, 990000}) accept(3, {sa_family=AF_INET, sin_port=htons(3436), sin_addr=inet_addr("192.168.1.20")}, [16]) = 14 fcntl64(14, F_GETFL) = 0x2 (flags O_RDWR) fcntl64(14, F_SETFL, O_RDWR|O_NONBLOCK) = 0 getpeername(14, {sa_family=AF_INET, sin_port=htons(3436), sin_addr=inet_addr("192.168.1.20")}, [16]) = 0 gettimeofday({1106285562, 261677}, NULL) = 0 fcntl64(14, F_SETFD, FD_CLOEXEC) = 0 select(15, [3 4 8 14], [], [3 4 8 14], {30, 0}) = 1 (in [14], left {30, 0}) recv(14, "GET /am/convertLinks2local HTTP/1.1\r\nHost: our.server.com:8080\r\nUser-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0\r\nAccept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,* /*;q=0.5\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip,deflate\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nKeep-Alive : 300\r\nConnection: keep-alive\r\nReferer: http://our.server.com:8080/am/convertLinks2local/ZPythonScriptHTML_editForm…: tree-s =\"eJzTyCkw5NLIKTDiClZ3hANXW3WuAmOuxEQ9AIOOB9Q\"\r\nAuthorization: Basic ZHN3aWZ0OjR0aGZsMDBy\r\n\r\n", 4096, 0) = 623 gettimeofday({1106285562, 264328}, NULL) = 0 kill(8744, SIGRTMIN) = 0 select(15, [3 4 8 14], [], [3 4 8 14], {30, 0}) = 0 (Timeout) rt_sigprocmask(SIG_SETMASK, NULL, [RTMIN], 8) = 0 rt_sigsuspend([] <unfinished ...> --- SIGRTMIN (Unknown signal 32) @ 0 (0) --- <... rt_sigsuspend resumed> ) = -1 EINTR (Interrupted system call) sigreturn() = ? (mask now [RTMIN]) select(15, [3 4 8 14], [], [3 4 8 14], {30, 0}) = 0 (Timeout) rt_sigprocmask(SIG_SETMASK, NULL, [RTMIN], 8) = 0 rt_sigsuspend([] <unfinished ...> --- SIGTERM (Terminated) @ 0 (0) --- <... rt_sigsuspend resumed> ) = -1 EINTR (Interrupted system call) getpid() = 8738 rt_sigaction(SIGTERM, {0x80e070, [], SA_RESTORER, 0x28da08}, {0x80e070, [], SA_RESTORER, 0x28da08}, 8) = 0 sigreturn() = ? (mask now [RTMIN]) rt_sigsuspend([] <unfinished ...> +++ killed by SIGKILL +++ You have new mail in /var/spool/mail/root [root@server Python-2.4]#

2 1

RE: [Zope] OFS.History (question 2)
by Pascal Peregrina 24 Jan '05

24 Jan '05

Thanks for all the answers. I managed to retrieve revision 1 and 2 of the subobject in the comparison method of the main object by using the methods defined in OFS.History and comparing _p_mtime. It works fine. Pascal -----Message d'origine----- De : Tino Wildenhain [mailto:tino@wildenhain.de] Envoyé : lundi 24 janvier 2005 19:57 À : Dieter Maurer Cc : Pascal Peregrina; 'zope(a)zope.org' Objet : Re: [Zope] OFS.History (question 2) Am Montag, den 24.01.2005, 19:27 +0100 schrieb Dieter Maurer: > Tino Wildenhain wrote at 2005-1-24 10:34 +0100: > > ... > >The only workaround I see right now is to get the list of historic > >versions for the subobject too, then select the version > >which timestamp is just below or equal to the timestamp of the > >container objects historical version. > > > >Maybe I add something like that to the historical patch for folders. > > I have something like a historical connection. > > It does not help with the history merging but it helps > to present a consistent view once you have chosen > a time you are interested in. > Sounds interesting! Where can I see it? :-) Regards Tino ********************************************************************** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept by MIMEsweeper for the presence of computer viruses. www.mimesweeper.com **********************************************************************

1 0

using mac xserves
by David Bear 24 Jan '05

24 Jan '05

I'm curious how many are using mac Xserves to run zope. Anyone know of statistics? -- David Bear phone: 480-965-8257 fax: 480-965-9189 College of Public Programs/ASU Wilson Hall 232 Tempe, AZ 85287-0803 "Beware the IP portfolio, everyone will be suspect of trespassing"

1 0

zope, wap and nationalities
by Jaroslav Lukesh 24 Jan '05

24 Jan '05

Dear Sirs, Do you have somebody tip for encoding national characters into wap? For example, character in latin-2 interpreted as "č" need to be interpreted in utf8 for wml as "č" I dont discover any python simple way how to recode these characters into wml, like <dtml-var "_.str.some_convert('string with ěščřž characters', iso8859-2, utf8_for_wap)"> ...or I am blind? Many thanks, -- Jaroslav Lukesh ----------------------------------------------------------- This e-mail can not contain any viruses because I use Linux

2 2

Making a self written module available to Zope
by Hong Yaun 23 Jan '05

23 Jan '05

If I have a self-written module/package which contains some module level methods, how can I make it available for use in the rest of Zope (e.g. in page templates, python script objects etc.)? Because it is just some utility functions, like the zope built-in sequence module, I don't want to register them as a zope class with all the manage* methods and create zope object from that. And I want the module to be initialized once, upon the start of zope or refresh of product maybe, to read some configuration file on the file system. What's the simplest way to achieve this? Greetings, Hong Yuan

4 4

Plagued by UnicodeEncodeError and UnicodeDecodeError
by davelists2＠peoplemerge.com 22 Jan '05

22 Jan '05

Hi, We have a ton of AT objects that we've imported from our databases, some of which are Japanese characters imported as unicode. In some cases, those could not be imported because of UnicodeEncodeErrors or UnicodeDecodeErrors loading the object from our DB or loading them into the ZODB. It would have been really nice to keep these entries, but we just put them in a try:except: block to get past them. Our current problem is that now we have the data in our DB, and they all display fine. But when we go to edit them through a base_edit and click Save, we're back to this: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 5: ordinal not in range(128). The traceback shows the field which is causing the error, and I don't see any unicode chars there! Zope 2.7.3-0 python 2.3.4 linux CMF 1.4.7 Plone 2.0.4 AT 1.3.1-final + MySqlSqlstorage

2 1

OFS.History
by Pascal Peregrina 22 Jan '05

22 Jan '05

Hi, I would like how to activate the History tab an a simple conent object that I wrote. Do I only need to make it inherit from Historical ? Pascal ********************************************************************** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept by MIMEsweeper for the presence of computer viruses. www.mimesweeper.com **********************************************************************

3 2

Where is Zope3X?
by Bobby Brewster 21 Jan '05

21 Jan '05

Postgresql 8.0 is out but where is Zope3X? It feels like forever, I am still waiting for Zope3X to start a database project based on Zope & PG. Should I forget about Zope3 for now and just use Zope 2.7? This really sucks. Shapehead ______________________________________________________________________ Post your free ad now! http://personals.yahoo.ca

8 12

Re: [Zope] Zcatalog searches
by Ken Ara 21 Jan '05

21 Jan '05

Brian, Some code you can use as a starting point to produce KWIC (key word in context) can be found here: http://zope.org/Members/Ioan/SiteSearch. It's old but works for me under Zope 2.7.3. This implementation requires the searchable text to be in Catalog metadata, which seems to be a bad thing, but I have never really understood just how bad.. The recent discussions about memory use have got me thinking again about this; I would appreciate any pointers to discussion/explanation of catalog metadata and memory. Best, Ken __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

5 5