- Zope - Zope lists

Re: [Zope] Running more than one instance on windows often block each other
by Tim Peters 03 Aug '05

03 Aug '05

[Sune Brøndum Wøller] > Thanks for the pointer. I have been debugging > select_trigger.py, and has some more info: > > The problem is that the call a.accept() sometimes hangs. > Apparently a.bind(self.address) allows us to bind to > a port that another zope instance already is bound to. > > The code creates the server socket a, and the client socket w, > and gets the client socket r by connecting w to a. Then it closes a. > a goes out of scope when __init__ terminates, and is probably garbage > collected at some point. Unless you're using a very old Python, `a` is collected before the call returns (where "the call" means the call of the function in which `a` is a local variable). Very old Pythons had an idiotic __del__ method attached to their Windows socket wrapper, which inhibited timely gc. > I tried moving the code to the following standalone script, and I can reproduce > the error with that. In the original code w is kept as an instance variable, and > r is passed to asyncore.dispatcher.__init__ and probably kept there. Yes, the socket bound to `r` also gets bound to `self.socket` by this call: asyncore.dispatcher.__init__ (self, r) > I simulate that by returning them, then the caller of socktest can keep them > around. > > I try to call socktest from different processes A and B (two pythons): > (w,r = socktest()) > The call in A gets port 19999. The second call, in B, either blocks, or takes > over port 19999 (I see the second process taking over the port in a port scanner.) Sorry, I can't reproduce this -- but you didn't give a test program, just an isolated function, and I'm not sure what you did with it. I called that function in an infinite loop, appending the return value to a global list, with a short (< 0.1 second) sleep between iterations, and closed the returned sockets fifty iterations after they were created. Ran that loop in two processes. No hangs, or any other oddities, for some minutes. It did _eventually_ hang-- and both processes at the same time --with netstat showing more than 4000 sockets hanging around in TIME_WAIT state then. I assume I bashed into some internal Windows socket resource limit there, which Windows didn't handle gracefully. Attaching to the processes under the MSVC 6 debugger, they were hung inside the MS socket libraries. Repeated this several times (everything appeared to work fine until > 4000 sockets were sitting in TIME_WAIT, and then both processes hung at approximately the same time). Concretely: sofar = [] try: while 1: print '.', stuff = socktest() # calling your function sofar.append(stuff) time.sleep(random.random()/10) if len(sofar) == 50: tup = sofar.pop(0) w, r = tup msg = str(random.randrange(1000000)) w.send(msg) msg2 = r.recv(100) assert msg == msg2, (msg, msg2) for s in tup: s.close() except KeyboardInterrupt: for tup in sofar: for s in tup: s.close() Note that there's also a bit of code there to verify that the connected sockets can communicate correctly; the `assert` never triggered. You haven't said which versions of Windows or Python you're using. I was using XP Pro SP2 and Python 2.3.5. Don't know whether either matters. It was certainly the case when I ran it that your > print port statement needed to display ports less than 19999 at times, meaning that the > a.bind((host, port)) did raise an exception at times. It never printed a port number less than 19997 for me. Did you ever see it print a port number less than 19999? > a.bind in B does not raise socket.error: (10048, 'Address already in use') as > expected, when the server socket in A is closed, even though the port is used by > the client socket r in A. I'm not sure what that's saying, but could be it's an illusion. For example, >>> import socket >>> s = socket.socket() >>> s.bind(('localhost', 19999)) >>> s.listen(2) >>> a1 = socket.socket() >>> a2 = socket.socket() >>> a1.connect(('localhost', 19999)) >>> a2.connect(('localhost', 19999)) >>> b1 = s.accept() >>> b2 = s.accept() >>> b1[0].getsockname() ('127.0.0.1', 19999) >>> b2[0].getsockname() ('127.0.0.1', 19999) >>> That is, it's normal for the `r` in > r, addr = a.accept() to repeat port numbers across multiple `accept()` calls, and indeed to duplicate the port number from the `bind` call. This always confused me (from way back in my Unix days -- it's not "a Windows thing"), and maybe it's not what you're talking about anyway. > If I remove a.close(), and keep a around (by passing it to the caller), a.bind > works as expected - it raises socket.error: (10048, 'Address already in use'). As above, I'm seeing `bind` raise exceptions regardless. > But in the litterature on sockets, I read it should be okay to close the server > socket and keep using the client sockets. > > So, is this a possible bug in bind() ? Sure feels that way to me, and I'm not seeing it (or don't know how to provoke it). But I'm not a socket expert, and am not sure I've ever met anyone who truly was ;-) > I have tested the new code from Tim Peters, it apparently works, ports are given > out by windows. > But could the same problem with bind occur here, since a is closed (and garbage > collected) ? (far less chance for that since we do not specify port numbers, I > know). > > I tried getting a pair of sockets with Tim's code, and then trying to bind a > third socket to the same port as a/r. And I got the same problem as above. Here I'm not sure what "the same problem" means, as you've described more than one problem. Do you mean that you get a hang? Or that you see suspiciously repeated port numbers? Or ...? Seeing concrete code might help. Last question for now: have you seen a hang on more than one flavor of Windows? Thanks for digging into this! [and Sune's code] > import socket, errno > > class BindError(Exception): > pass > > > def socktest(): > """blabla > """ > > address = ('127.9.9.9', 19999) > > a = socket.socket (socket.AF_INET, socket.SOCK_STREAM) > w = socket.socket (socket.AF_INET, socket.SOCK_STREAM) > > # set TCP_NODELAY to true to avoid buffering > w.setsockopt(socket.IPPROTO_TCP, 1, 1) > > # tricky: get a pair of connected sockets > host='127.0.0.1' > port=19999 > > while 1: > print port > try: > a.bind((host, port)) > break > except: > if port <= 19950: > raise BindError, 'Cannot bind trigger!' > port=port - 1 > > a.listen (1) > w.setblocking (0) > try: > w.connect ((host, port)) > except: > pass > r, addr = a.accept() > a.close() > w.setblocking (1) > > #return (a, w, r) > return (w, r) > #return w

2 20

RE: [Zope] zserver blocking behavior with a slow request
by Jim Abramson 03 Aug '05

03 Aug '05

> > I'm finding that once a user requests a page which runs a > particularly slow sql (say up to 5 minutes), > any other > subsequent requests seem to take the hit as well, and return > very slowly. > Take a look at DadlockDebugger product - you'll see what is > happening with your Zope threads. > Thanks, this looks quite handy, I am going to give it a whirl (I assume that threadframe module dependency hasn't yet hosed anyone's Linux machine - funny how a disclaimer can inspire paranoia) > Isn't it better idea to execute long running query outside of > zope - external method or sth, and only check for results > even with simple page reload? We did something like this with > pdf generation and this worked as expected, but of course you > need some additional work to create such thing. > Yes I agree, in my case those SELECTs aren't *supposed* to be taking 5 minutes. That's a separate problem. However when such a problem bites I don't want all of my users to suffer, just the unlucky one who requested the doomed sql to be run. > > So is this blocking effect just expected behavior for zope? > We are using Zope and Oracle too, and even 10 threads... and > I didn't noticed something like this. What is your processor > doing then? What is the CPU(s?) load. The single CPU isn't loaded at all. But if DeadlockDebugger doesn't turn up any good clues I will do some investigating at the system level. Thanks for advice, Jim

1 0

PCNG 1.2.7 does exception when being added.
by Vlada Macek 03 Aug '05

03 Aug '05

I have the following configuration, both Zope and Plone are up to date to these branches: svn://svn.zope.org/repos/main/Zope/branches/Zope-2_8-branch https://svn.plone.org/svn/plone/bundles/2.1 Plone version: Plone 2.1-beta2 (svn/unreleased) Zope version: Zope (Zope 2.8.1-b1, python 2.3.5, linux2) Python 2.3.5 (#2, May 4 2005, 08:51:39) [GCC 3.3.5 (Debian 1:3.3.5-12 Browser(s) and version(s): Firefox 1.0.4 Operating system: Linux Debian 3.1 (Sarge) stable When I install PloneCollectorNG 1.2.5 from tar.gz and try to add the "pcng tracker" from Plone, the following error will always rise: Error Type BadRequest Error Value The id "plaintext_lexicon" is invalid - it is already in use. The full log item from the Plone Error Log is attached. Is it that PCNG 1.2.7 cannot be used with recent SVN Zope/Plone? If so, couldn't this be mentioned somewhere in the docs, please? Thanks in advance. -- \//\/\ (Sometimes credited as 1494 F8DD 6379 4CD7 E7E3 1FC9 D750 4243 1F05 9424.) Čas 2005/08/03 18:45:45.912 GMT+2 Jméno uživatele admin (admin) Adresa URL požadavku http://localhost:8080/HIS/sczescwerheerhdfb/createObject Typ výjimky BadRequest Hodnota výjimky The id "plaintext_lexicon" is invalid - it is already in use. Traceback (innermost last): * Module ZPublisher.Publish, line 113, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 40, in call_object * Module Products.CMFFormController.FSControllerPythonScript, line 105, in __call__ * Module Products.CMFFormController.Script, line 141, in __call__ * Module Products.CMFCore.FSPythonScript, line 108, in __call__ * Module Shared.DC.Scripts.Bindings, line 311, in __call__ * Module Shared.DC.Scripts.Bindings, line 348, in _bindAndExec * Module Products.CMFCore.FSPythonScript, line 163, in _exec * Module None, line 16, in createObject <FSControllerPythonScript at /HIS/createObject used for /HIS/sczescwerheerhdfb> Line 16 * Module Products.ATContentTypes.lib.constraintypes, line 274, in invokeFactory * Module Products.CMFCore.PortalFolder, line 408, in invokeFactory * Module Products.CMFCore.TypesTool, line 930, in constructContent * Module Products.CMFCore.TypesTool, line 340, in constructInstance * Module Products.CMFCore.TypesTool, line 571, in _constructInstance * Module Products.PloneCollectorNG.Collector, line 4, in addPloneCollectorNG * Module OFS.ObjectManager, line 293, in _setObject * Module Products.PloneCollectorNG.Collector, line 122, in manage_afterAdd * Module Products.PloneCollectorNG.Collector, line 146, in setup_tools * Module Products.PloneCollectorNG.Collector, line 162, in _setup_catalog * Module Products.BTreeFolder2.BTreeFolder2, line 431, in _setObject * Module Products.PloneCollectorNG.Collector, line 952, in manage_afterAdd * Module Products.CMFCore.CatalogTool, line 184, in _initIndexes * Module OFS.ObjectManager, line 265, in _setObject * Module OFS.ObjectManager, line 84, in checkValidId BadRequest: The id "plaintext_lexicon" is invalid - it is already in use. REQUEST form type_name 'PloneCollectorNG' cookies tree-s 'eJzTyCkw5NLIKTDiClZ3hANXW3WuAmOuRKCECUjWFEnWxzsQJGvGlQiSNkfR5wuSsQDKJCbqAQD1tBZ8' zmi_top_frame '' zmi_use_css '1' I18N_LANGUAGE 'cs' dtpref_cols '95%' __ac_name 'admin' wstyle '' dtpref_rows '25' photo_display 'large' _ZopeId '40204748A18SzYLK0d0' lazy items other zmi_top_frame '' tree-s 'eJzTyCkw5NLIKTDiClZ3hANXW3WuAmOuRKCECUjWFEnWxzsQJGvGlQiSNkfR5wuSsQDKJCbqAQD1tBZ8' URL3 'http://localhost:8080' URL2 'http://localhost:8080/HIS' AUTHENTICATION_PATH '' LANGUAGE 'cs' AUTHENTICATED_USER admin type_name 'PloneCollectorNG' SERVER_URL 'http://localhost:8080' traverse_subpath [] SESSION id: 11230875371247919349, token: 40204748A18SzYLK0d0, contents: [] dtpref_cols '95%' __ac_name 'admin' dtpref_rows '25' ACTUAL_URL 'http://localhost:8080/HIS/sczescwerheerhdfb/createObject' _ZopeId '40204748A18SzYLK0d0' URL 'http://localhost:8080/HIS/sczescwerheerhdfb/createObject' I18N_LANGUAGE 'cs' wstyle '' PUBLISHED <FSControllerPythonScript at /HIS/createObject used for /HIS/sczescwerheerhdfb> controller_state <Products.CMFFormController.ControllerState.ControllerState object at 0x4226364c> TraversalRequestNameStack [] LANGUAGE_TOOL <Products.PloneLanguageTool.LanguageTool.LanguageBinding instance at 0x4488c90c> zmi_use_css '1' BASE1 'http://localhost:8080' BASE2 'http://localhost:8080/HIS' BASE3 'http://localhost:8080/HIS/sczescwerheerhdfb' BASE4 'http://localhost:8080/HIS/sczescwerheerhdfb/createObject' URL1 'http://localhost:8080/HIS/sczescwerheerhdfb' photo_display 'large' URL0 http://localhost:8080/HIS/sczescwerheerhdfb/createObject URL1 http://localhost:8080/HIS/sczescwerheerhdfb URL2 http://localhost:8080/HIS URL3 http://localhost:8080 BASE0 http://localhost:8080 BASE1 http://localhost:8080 BASE2 http://localhost:8080/HIS BASE3 http://localhost:8080/HIS/sczescwerheerhdfb BASE4 http://localhost:8080/HIS/sczescwerheerhdfb/createObject environ HTTP_COOKIE '__ac_name="admin"; zmi_top_frame=""; dtpref_cols="95%"; dtpref_rows="25"; photo_display=large; zmi_use_css="1"; wstyle=; tree-s="eJzTyCkw5NLIKTDiClZ3hANXW3WuAmOuRKCECUjWFEnWxzsQJGvGlQiSNkfR5wuSsQDKJCbqAQD1tBZ8"; _ZopeId="40204748A18SzYLK0d0"; I18N_LANGUAGE="cs"' SERVER_SOFTWARE 'Zope/(Zope 2.8.1-b1, python 2.3.5, linux2) ZServer/1.1 Plone/Unknown' SCRIPT_NAME '' REQUEST_METHOD 'GET' HTTP_KEEP_ALIVE '300' SERVER_PROTOCOL 'HTTP/1.1' QUERY_STRING 'type_name=PloneCollectorNG' channel.creation_time 1123087523 CONNECTION_TYPE 'keep-alive' HTTP_ACCEPT_CHARSET 'ISO-8859-2,utf-8;q=0.7,*;q=0.7' HTTP_USER_AGENT 'Mozilla/5.0 (X11; U; Linux i686; cs-CZ; rv:1.7.8) Gecko/20050517 Firefox/1.0.4 (Debian package 1.0.4-2)' HTTP_REFERER 'http://localhost:8080/HIS/sczescwerheerhdfb' SERVER_NAME '0.0.0.0' REMOTE_ADDR '127.0.0.1' PATH_TRANSLATED '/HIS/sczescwerheerhdfb/createObject' SERVER_PORT '8080' HTTP_HOST 'localhost:8080' HTTP_ACCEPT 'text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5' GATEWAY_INTERFACE 'CGI/1.1' HTTP_ACCEPT_LANGUAGE 'cs,en-us;q=0.7,en;q=0.3' HTTP_ACCEPT_ENCODING 'gzip,deflate' PATH_INFO '/HIS/sczescwerheerhdfb/createObject'

4 3

RE: [Zope] Mount points
by Pascal Peregrina 03 Aug '05

03 Aug '05

Ok will do. Thanks -----Original Message----- From: zope-bounces(a)zope.org [mailto:zope-bounces@zope.org] On Behalf Of Dieter Maurer Sent: 03 August 2005 18:36 To: Pascal Peregrina Cc: zope(a)zope.org Subject: Re: [Zope] Mount points Pascal Peregrina wrote at 2005-8-2 19:39 +0100: >Is there a way to automount clientstorages ? >(I mean by just editing zope.conf and restarting the Zope instance) No >Currently I have to add a ZODB Mount Point object through ZMI... You can automate this mount point creation in a script of yourself... -- Dieter _______________________________________________ Zope maillist - Zope(a)zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) ********************************************************************** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept by MIMEsweeper for the presence of computer viruses. www.mimesweeper.com **********************************************************************

1 0

Mount points
by Pascal Peregrina 03 Aug '05

03 Aug '05

Hi, Is there a way to automount clientstorages ? (I mean by just editing zope.conf and restarting the Zope instance) Currently I have to add a ZODB Mount Point object through ZMI... Thanks (I use Zope 2.8.0 / python 2.3.5) Pascal ********************************************************************** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept by MIMEsweeper for the presence of computer viruses. www.mimesweeper.com **********************************************************************

2 1

test for the existence of a nodeValue
by Jon Bowlas 03 Aug '05

03 Aug '05

Hi All, I'm pretty new to python and zope but would like to know how I can adapt this script below to test to see if nodeValue actually contains a value and if it doesn't then I would like to return a default value I can check for using my zpt that references this script: if not attobject: return '' version = attobject.get_viewable() nodes = version.content.documentElement.getElementsByTagName(attname) if not nodes: return '' return nodes[0].childNodes[0].nodeValue Two parameters are passed to this script attobject, attname. 'attobject' is an object id and 'attname' is the childNode with the nodeValue I wish to return. Hope someone can help, and I hope this makes sense. Thanks Jon

2 1

zserver blocking behavior with a slow request
by Jim Abramson 03 Aug '05

03 Aug '05

On a production Zope (2.7.6) running behind apache and connected to an Oracle database, I'm finding that once a user requests a page which runs a particularly slow sql (say up to 5 minutes), any other subsequent requests seem to take the hit as well, and return very slowly. I have 8 zserver threads and a ZODB pool size of 10. I'm fairly certain I'm not maxing these out. So is this blocking effect just expected behavior for zope? One point of note is that the requests passed from apache all bear the remote ip of 127.0.0.1. Could zserver be throttling the incoming requests due to their identical REMOTE_IP (or some other apache configuration detail)? Is there anything else I should investigate? OH btw, I am using my own connection pool and managing db interface / transactions manually, so ZSQL should not be considered a culprit. Furthermore I'm watching the connections/pool themselves and they are not maxed either. Thanks for any help. Jim

4 3

When "Debug Spinning Zope" doesn't work
by Paul Winkler 03 Aug '05

03 Aug '05

It used to be that on the rare occasions that Zope hung, I could use the recipe at http://zopelabs.com/cookbook/1073504990 to diagnose the problem (usually my own code was at fault). But, what does it mean when I get nothing at all from "info threads" in gdb? I get this on two different systems: system A: python2.3.5 zope 2.7.7 gdb 5.2-2 system B: python 2.3.4 zope 2.7.3 gdb 5.2-2 Lately this seems to happen regardless of whether Zope is responsive or not. e.g.: $ cat Z2.pid $ cat /zope/ZopeInstanceHome/var/Z2.pid 24161 $ sudo gdb python2.3 Password: GNU gdb Red Hat Linux (5.2-2) Copyright 2002 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux"... (gdb) attach 24161 Attaching to program: /usr/local/bin/python2.3, process 24161 Reading symbols from /lib/libpthread.so.0...done. (snip copious output) Reading symbols from /usr/local/lib/python2.3/lib-dynload/itertools.so...done. Loaded symbols for /usr/local/lib/python2.3/lib-dynload/itertools.so 0x080775af in PyDict_GetItem (op=0x44d16934, key=0x401ae020) at Objects/dictobject.c:498 498 return (mp->ma_lookup)(mp, key, hash)->me_value; (gdb) info threads {gdb) That's it, nothing. It used to work for me. Any clues? -- Paul Winkler http://www.slinkp.com

3 3

RE: [Zope] How to instruct my Zope to use another pytrhon than the oneit was budled with.
by Jim Abramson 03 Aug '05

03 Aug '05

> > I also have another Python version on my system and i was > wondering how could i "tell" Zope to use that one instead of > the one it comes with from the installation executable package. > If you start up using the runzope.bat script in ZOPE_HOME/bin, just edit the value of PYTHON to point to the desired alternate executable. Don't automatically expect things to work smoothly on the first try - watch the output from startup and see if you need to pull missing dependencies into the sys.path (edit the value of PYTHONPATH if this is necessary) - or if the executable is even compatible at all (version, distribution/standard library, etc). Jim

1 0

Re: [Zope] RE: Blobs in Interbase
by Ashley Lloyd 03 Aug '05

03 Aug '05

Bob, you're a star! I unpacked it, restarted Zope, and there it was! Marvellous, I've now got a kinterbase installed that works. Thats the good news. The bad news is I still can't update the blob field - I still get errors. I think, unless anyone here has any more ideas, I'll have to go and bother people in another mailing list. I've tried absolutely everything I can think of - sqlvar tags of type string and nb, trying cast(... as blob) in the sql (doesn't seem to work), I've even just tried setting it to 'Hello world', without using any dtml variables at all, and it won't accept it. >From what I've read elsewhere, there are tools in some software (mostly for Delphi I believe), that will allow updates of text blob fields, so I was hoping that kinterbasdb had something in it as well, but if it does, it isn't working on mine!! The interbase version is 6.0-6.1, if that jogs any memories for anyone. Thanks once again to everyone who has tried to help me. Kind regards Ashley Ashley Lloyd wrote: Hi Bob, If you don't mind doing that, I'd really appreciate it. Nothing I seem to be trying to install at the moment seems to be working. As with Phil, mine is from a Plone install, but I have got other versions of Python elsewhere, (I think there's even an old Zope standalone instance still lurking somewhere, altho I don't think it can run) just to confuse matters. Anyway, my Zope is even older - 2.6.1, so hopefully it'll be compatible. Thanks very much again for your help. Ashley ----- Original Message ----- From: "Bob Corriher" <bobc(a)p-wave.com> To: <phil(a)xfr.co.uk> Cc: "Ashley Lloyd" <ashleylloyd(a)hotmail.com>; <zope(a)zope.org> Sent: Tuesday, August 02, 2005 4:01 PM Subject: Re: [Zope] RE: Blobs in Interbase Hi Phil & Ashley, I also have kinterbasedbDA working on Windows. It is with an older version of Zope (2.6.4). The DA version is 1.1.1.1. The kinterbasdb version is 3.0.1. It uses a dll called _kinterbasedb.dll, instead of a pyd. I don't remember the exact installation process anymore, but I think I just grabbed it from another working site. If you like, I could zip it up and send it to you. The site uses blobs (all sub_type text) in many places, so maybe older/other versions don't support this as well. Bob Corriher CTO P-Wave Inc. Philip Kilner wrote: Hi Ashley, Can't help you on the blob front... Ashley Lloyd wrote: I've had trouble installing kinterbase (it complains about LDAP..?), so at the moment I use gvib. I've pencilled in quite a bit of time today to try to get kinterbase installed. Unfortunately I'm on Windows, which I'm guessing contributes to the installation problems! ...but I can confirm that KIinterbaseDA works nicely on Windows - I've been happily using it for some time, and haven't had any issues at all (BTW, I don't use LDAP and KInterbaseDA has never mentioned it!). The only problem I had at install time was getting my head around the distinction between the Python part and the Zope DA part. HTH -- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.0.338 / Virus Database: 267.9.8/61 - Release Date: 8/1/2005 ------------------------------------------------------------------------------ No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.0.338 / Virus Database: 267.9.8/61 - Release Date: 8/1/2005

2 3