March 2001 - Zope - Zope lists

RE: DTML documents killing Zope (was RE: [Zope] ZServer dying)
by Mayers, Philip J 28 Mar '01

28 Mar '01

Yes, I'm running a stock 2.3.0 installation *except* I made that one change to the sourcecode (otherwise the PythonScripts would take a long time to load ever 10-30 seconds. The line used to read: if getattr(self, Python_magic, None) != Python_magic: And it now reads: if getattr(self, 'Python_magic', None) != Python_magic: FYI - it appears to be the _number_ of parameters. If I call the DTML document with 12 parameters (context,REQUEST,p0=0,...p11=11) it fails under load, anything less than that (i.e. drop *any* one or more of the paramters) and it's ok. I don't know if it's some critical limit I'm hitting (like stack space or something equivalent) - I still don't know enough about Zope's internals to find out. (If you want a rootshell on the machine running it, let me know) Regards, Phil +----------------------------------+ | Phil Mayers, Network Support | | Centre for Computing Services | | Imperial College | +----------------------------------+ -----Original Message----- From: Evan Simpson [mailto:evan@4-am.com] Sent: 28 March 2001 20:22 To: Mayers, Philip J Cc: zope(a)zope.org Subject: Re: DTML documents killing Zope (was RE: [Zope] ZServer dying) From: "Mayers, Philip J" <p.mayers(a)ic.ac.uk> > I have a PythonScript (called as a Web page) which renders the HTML by > calling external DTML documents, and supplies form field variables as 10 > keyword arguments. This seems to crash Zope if called under load. If I don't > supply the keyword arguments (and of course, the field isn't filled in) then > everything is OK. Just to get this out of the way as a possible cause, if you grep PythonScript.py for "magic", does the following line have quote marks around the first 'Python_magic' (in getattr)? It should. if getattr(self, 'Python_magic', None) != Python_magic: Cheers, Evan @ digicool

2 1

DTML documents killing Zope (was RE: [Zope] ZServer dying)
by Mayers, Philip J 28 Mar '01

28 Mar '01

Ok, I've narrowed down what is killing my Zope using the following script: import threading import httplib import sys from base64 import encodestring import string import time def getfunc(server,url,user=None,pw=None): h = httplib.HTTP(server) h.putrequest("GET",url) if user and pw: try: password = encodestring("%s:%s" % (user,pw)) password = string.replace(password,"\012","") h.putheader("AUTHORIZATION","Basic %s" % password) except: pass h.endheaders() errcode,errmsg,headers = h.getreply() if errcode != 200: print "failed: %s" % (errmsg,) print "Done thread at %s" % (time.time(),) threads = [] num_threads = 10 try: num_threads = string.atoi(sys.argv[5]) except: pass for i in range(1,num_threads): t = threading.Thread(None,getfunc,None,tuple(sys.argv[1:5]),{}) threads.append(t) t.start() for t in threads: t.join() print "%s joined at %s" % (t.getName(),time.time()) Fun for all the benchmarking family. I can do this: python floodget.py hostname:8080 /path/to/page username password 1000 And it works. However, python floodget.py hostname:8080 '/path/to/page?arg1=a&arg2=b' username password 1000 Fails very quickly. Now, the page in question is a PythonScript (hmm...) and the code that is executed when the arguments are passed renders a DTML comment using: print container.folder.dtml_method(context,context,REQUEST,key1=arg1,key2=arg2) And so on... If I comment out the above line, *it all works under load* - that is to say, it's apparently the rendering of the DTML document that is killing things... In fact, I just did some more diagnosis. It's only when I pass a lot of keyword arguments - I'm working on the threshold that triggers it now... This is all very unfortunate. So - to summarise: I have a PythonScript (called as a Web page) which renders the HTML by calling external DTML documents, and supplies form field variables as 10 keyword arguments. This seems to crash Zope if called under load. If I don't supply the keyword arguments (and of course, the field isn't filled in) then everything is OK. Weird huh? I can repro this reliably - how can I debug and diagnose it? Sven, does this fit your experience? Regards, Phil +----------------------------------+ | Phil Mayers, Network Support | | Centre for Computing Services | | Imperial College | +----------------------------------+ -----Original Message----- From: Mayers, Philip J [mailto:p.mayers@ic.ac.uk] Sent: 28 March 2001 14:18 To: 'Chris Withers'; Sven Cc: zope(a)zope.org Subject: RE: [Zope] ZServer dying No, I run: su -l zope -c 'nohup /usr/local/Zope/start -X -w 8080 &' I'm stumped. Sven, are you running Postgresql? If so, what version? Next time it dies and the process is there, try strace'ing it with "strace -p pid -d" Regards, Phil +----------------------------------+ | Phil Mayers, Network Support | | Centre for Computing Services | | Imperial College | +----------------------------------+ -----Original Message----- From: Chris Withers [mailto:chrisw@nipltd.com] Sent: 28 March 2001 13:48 To: Sven Cc: zope(a)zope.org Subject: Re: [Zope] ZServer dying Sven wrote: > > so i just have to restart zope, and some other times, it freezes but i still > can see the processes (then killal and restart). > > my developpement version does this between 1 and 10 times a day (redhat 7 / > zope 2.3.0 too) Do you run both setups with the -D option? cheers Chris _______________________________________________ Zope maillist - Zope(a)zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev ) _______________________________________________ Zope maillist - Zope(a)zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )

2 1

Re: [Zope] LOCK/UNLOCK
by Fernstrom, Christer 28 Mar '01

28 Mar '01

Looking through the mail archives I realise that I'm not the only person who has questions about locking in ZOPE. I know that you can use versions, and as long as every concurrent user does updates in a separate version, locks are set, visible to all, and prevents accidental overwrites. Problem is that users who are just content providers (not programmers) have to explicitly enter versions, then commit changes they've done in the versions. This is far from ideal. I haven't found parts of the ZOPE API that deals with handling versions, but I was playing with the idea to provide a "modify" button on the pages that would: - create a personal version for the user (unless one is already there) - start using the version - go to the manage_main for the page to allow updates - upon SAVE (hm, how do I catch this?), commit the version changes (ie do version "save") - redisplay the rendered page Looks simple on the paper. Has anybody done something like this? Would it work? christer christer.fernstrom(a)xrce.xerox.com >I am totaly baffeled by the correct usage of LOCKs in ZOPE. >Example please? > >Thanks, >-- >Tim Cook, President - FreePM,Inc. >http://www.FreePM.com Office: (901) 884-4126 >ONLINE DEMO: http://www.freepm.org:8080/FreePM

2 2

RE: DTML documents killing Zope (was RE: [Zope] ZServer dying)
by Mayers, Philip J 28 Mar '01

28 Mar '01

Calling the DTML document with 2 positional parameters and 10 keyword args will kill it after 5-10 requests - 2 positional and 9 keyword will not. Very odd... Regards, Phil +----------------------------------+ | Phil Mayers, Network Support | | Centre for Computing Services | | Imperial College | +----------------------------------+ -----Original Message----- From: Mayers, Philip J [mailto:p.mayers@ic.ac.uk] Sent: 28 March 2001 19:55 To: 'Chris Withers'; Sven Cc: zope(a)zope.org Subject: DTML documents killing Zope (was RE: [Zope] ZServer dying) Ok, I've narrowed down what is killing my Zope using the following script: import threading import httplib import sys from base64 import encodestring import string import time def getfunc(server,url,user=None,pw=None): h = httplib.HTTP(server) h.putrequest("GET",url) if user and pw: try: password = encodestring("%s:%s" % (user,pw)) password = string.replace(password,"\012","") h.putheader("AUTHORIZATION","Basic %s" % password) except: pass h.endheaders() errcode,errmsg,headers = h.getreply() if errcode != 200: print "failed: %s" % (errmsg,) print "Done thread at %s" % (time.time(),) threads = [] num_threads = 10 try: num_threads = string.atoi(sys.argv[5]) except: pass for i in range(1,num_threads): t = threading.Thread(None,getfunc,None,tuple(sys.argv[1:5]),{}) threads.append(t) t.start() for t in threads: t.join() print "%s joined at %s" % (t.getName(),time.time()) Fun for all the benchmarking family. I can do this: python floodget.py hostname:8080 /path/to/page username password 1000 And it works. However, python floodget.py hostname:8080 '/path/to/page?arg1=a&arg2=b' username password 1000 Fails very quickly. Now, the page in question is a PythonScript (hmm...) and the code that is executed when the arguments are passed renders a DTML comment using: print container.folder.dtml_method(context,context,REQUEST,key1=arg1,key2=arg2) And so on... If I comment out the above line, *it all works under load* - that is to say, it's apparently the rendering of the DTML document that is killing things... In fact, I just did some more diagnosis. It's only when I pass a lot of keyword arguments - I'm working on the threshold that triggers it now... This is all very unfortunate. So - to summarise: I have a PythonScript (called as a Web page) which renders the HTML by calling external DTML documents, and supplies form field variables as 10 keyword arguments. This seems to crash Zope if called under load. If I don't supply the keyword arguments (and of course, the field isn't filled in) then everything is OK. Weird huh? I can repro this reliably - how can I debug and diagnose it? Sven, does this fit your experience? Regards, Phil +----------------------------------+ | Phil Mayers, Network Support | | Centre for Computing Services | | Imperial College | +----------------------------------+ -----Original Message----- From: Mayers, Philip J [mailto:p.mayers@ic.ac.uk] Sent: 28 March 2001 14:18 To: 'Chris Withers'; Sven Cc: zope(a)zope.org Subject: RE: [Zope] ZServer dying No, I run: su -l zope -c 'nohup /usr/local/Zope/start -X -w 8080 &' I'm stumped. Sven, are you running Postgresql? If so, what version? Next time it dies and the process is there, try strace'ing it with "strace -p pid -d" Regards, Phil +----------------------------------+ | Phil Mayers, Network Support | | Centre for Computing Services | | Imperial College | +----------------------------------+ -----Original Message----- From: Chris Withers [mailto:chrisw@nipltd.com] Sent: 28 March 2001 13:48 To: Sven Cc: zope(a)zope.org Subject: Re: [Zope] ZServer dying Sven wrote: > > so i just have to restart zope, and some other times, it freezes but i still > can see the processes (then killal and restart). > > my developpement version does this between 1 and 10 times a day (redhat 7 / > zope 2.3.0 too) Do you run both setups with the -D option? cheers Chris _______________________________________________ Zope maillist - Zope(a)zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev ) _______________________________________________ Zope maillist - Zope(a)zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev ) _______________________________________________ Zope maillist - Zope(a)zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )

1 0

Counter Rentrancy
by David Kyte 28 Mar '01

28 Mar '01

Hi, If I have a DTML document for generating and tracking say order numbers, is the code rentrant/Atomic? <dtml-call "REQUEST.set('counter', counter + 1)"> <dtml-call "manage_changeProperties( counter=REQUEST['counter'] )"> <dtml-return counter > What I'm really asking is how many threads run in parallel, and how would multiple threads accessing this code react. Do I need some form of transation or locking? Thanks in anticipation David Kyte

4 4

Re: [Zope] resolving between like-named parent folders?
by Oliver Bleutgen 28 Mar '01

28 Mar '01

> How would one go about the following? > I have a folder hierarchy represented as shown: > A > B > index_html > C > D > A > B > index_html > I would like the C.D.A.B.index_html to redirect to A.B.index_html You can use <dtml-var "RESPONSE.redirect(PARENTS[-1].A.B.absolute_url())"> There are some really good explanations of PARENTS out there if you're interested. cheers, oliver

1 0

Re: [Zope] ParsedXML and zope2.3.1b3
by Karl Anderson 28 Mar '01

28 Mar '01

bak (kedai) <kedai(a)kedai.com.my> writes: > hi > i can't add any ParsedXml instance. i got a name error ParsedXML isn't compatable with Zope 2.3.1b3 because of an acquisition bug. Use Zope 2.3.1b2 or earlier, or a later Zope release when one comes out, or a Zope checkout from after Mar 27 2001, and it should work fine. -- Karl Anderson karl(a)digicool.com

1 0

Re: [Zope] Can ZClass and ZCatalog do this?
by Dieter Maurer 28 Mar '01

28 Mar '01

Brad Clements writes: > ... need for advanced sorting ... Oleg Broytmann wants to implement an extension to the "in" command that would provide his advanced sorting. Until it is implemented, you can do it yourself in an External Method or (maybe) a Python Script, using Python's built-in "sort" method. Search for the thread (very new) to get some ideas. Dieter

1 0

Re: [Zope] Performance under load using SQL (Postgresql)
by Dieter Maurer 28 Mar '01

28 Mar '01

Mayers, Philip J writes: > ... > I'm using Postgresql (makes no different whether it's 7.0 or the 7.1RC1) > with ZPoPyDA and/or psycopg - they both suffer from the problem. What > happens is that the child threads of ZServer die, leaving me with the > top-level process (doing a waitpid on it's child) and the child (doing a > select on the PCGI socket) Usually, they say something (--> log file, enabled with "STUPID_LOG_FILE=....") when they die. This may give you a hint. Often they produce a "core" file, when they die. However, some configuration experts disable the core generation. Reenable them and see whether you get one. Then you can look into them with a debugger. Dieter

1 0

Re: [Zope] Tree component
by Dieter Maurer 28 Mar '01

28 Mar '01

After Dark writes: > Hello, I need to make a page that allow to > insert/delete/update archives from zope. For that I will > need to have a tree of the objects(like directories and > subdirectories). > > I want to know if there is any example of that or how can I > do it. Where are your archived documents: in ZODB, in a relational database, in the file system? How is the hierachical structure defined? There is a HowTo on advanced "tree" use on zope.org. Maybe, you look at it. Dieter

1 0