DTML documents killing Zope (was RE: [Zope] ZServer dying)

Wed, 28 Mar 2001 19:54:33 +0100

Ok, I've narrowed down what is killing my Zope using the following script:

import threading
import httplib
import sys
from base64 import encodestring
import string
import time

def getfunc(server,url,user=None,pw=None):
        h = httplib.HTTP(server)
        h.putrequest("GET",url)
        if user and pw:
                try:
                        password = encodestring("%s:%s" % (user,pw))
                        password = string.replace(password,"\012","")
                        h.putheader("AUTHORIZATION","Basic %s" % password)
                except:
                        pass
        h.endheaders()
        errcode,errmsg,headers = h.getreply()
        if errcode != 200:
                print "failed: %s" % (errmsg,)
        print "Done thread at %s" % (time.time(),)

threads = []
num_threads = 10
try:
        num_threads = string.atoi(sys.argv[5])
except:
        pass

for i in range(1,num_threads):
        t = threading.Thread(None,getfunc,None,tuple(sys.argv[1:5]),{})
        threads.append(t)
        t.start()
for t in threads:
        t.join()
        print "%s joined at %s" % (t.getName(),time.time())

Fun for all the benchmarking family. I can do this:

  python floodget.py hostname:8080 /path/to/page username password 1000

And it works. However,

  python floodget.py hostname:8080 '/path/to/page?arg1=a&arg2=b' username
password 1000

Fails very quickly. Now, the page in question is a PythonScript (hmm...) and
the code that is executed when the arguments are passed renders a DTML
comment using:

  print
container.folder.dtml_method(context,context,REQUEST,key1=arg1,key2=arg2)

And so on... If I comment out the above line, *it all works under load* -
that is to say, it's apparently the rendering of the DTML document that is
killing things... In fact, I just did some more diagnosis. It's only when I
pass a lot of keyword arguments - I'm working on the threshold that triggers
it now... This is all very unfortunate.

So - to summarise:

I have a PythonScript (called as a Web page) which renders the HTML by
calling external DTML documents, and supplies form field variables as 10
keyword arguments. This seems to crash Zope if called under load. If I don't
supply the keyword arguments (and of course, the field isn't filled in) then
everything is OK.

Weird huh? I can repro this reliably - how can I debug and diagnose it?
Sven, does this fit your experience?

Regards,
Phil

+----------------------------------+
| Phil Mayers, Network Support     |
| Centre for Computing Services    |
| Imperial College                 |
+----------------------------------+  

-----Original Message-----
From: Mayers, Philip J [mailto:p.mayers@ic.ac.uk]
Sent: 28 March 2001 14:18
To: 'Chris Withers'; Sven
Cc: zope@zope.org
Subject: RE: [Zope] ZServer dying

No, I run:

su -l zope -c 'nohup /usr/local/Zope/start -X -w 8080 &'

I'm stumped. Sven, are you running Postgresql? If so, what version?

Next time it dies and the process is there, try strace'ing it with "strace
-p pid -d"

Regards,
Phil

+----------------------------------+
| Phil Mayers, Network Support     |
| Centre for Computing Services    |
| Imperial College                 |
+----------------------------------+  

-----Original Message-----
From: Chris Withers [mailto:chrisw@nipltd.com]
Sent: 28 March 2001 13:48
To: Sven
Cc: zope@zope.org
Subject: Re: [Zope] ZServer dying

Sven wrote:
> 
> so i just have to restart zope, and some other times, it freezes but i
still
> can see the processes (then killal and restart).
> 
> my developpement version does this between 1 and 10 times a day (redhat 7
/
> zope 2.3.0 too)

Do you run both setups with the -D option?

cheers

Chris

_______________________________________________
Zope maillist  -  Zope@zope.org
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )