August 2005 - Zope - Zope lists

NauSchduler problem
by Barry Drake 03 Aug '05

03 Aug '05

I'm using NauSchduler to automate sending a news digest out twice daily to users of our site. The digest simply has the Subject of the news item and the URL to item on our site. The Script Python that sends the digest works fine when tested alone, but when I run it from NS the URL, gotten from obj.absolute_url(), is replace with the physical path. I see many calls to getPhysicalPath() in the NS source. But, I don't understand why it would overwrite my URL. Has anyone else seen this behavior? BTW, this is the first problem I've had with this product. It's been great for other scheduled tasks. Barry Drake

2 2

Re: [Zope] Running more than one instance on windows often block each other
by Tim Peters 03 Aug '05

03 Aug '05

[Sune Brøndum Wøller] > Thanks for the pointer. I have been debugging > select_trigger.py, and has some more info: > > The problem is that the call a.accept() sometimes hangs. > Apparently a.bind(self.address) allows us to bind to > a port that another zope instance already is bound to. > > The code creates the server socket a, and the client socket w, > and gets the client socket r by connecting w to a. Then it closes a. > a goes out of scope when __init__ terminates, and is probably garbage > collected at some point. Unless you're using a very old Python, `a` is collected before the call returns (where "the call" means the call of the function in which `a` is a local variable). Very old Pythons had an idiotic __del__ method attached to their Windows socket wrapper, which inhibited timely gc. > I tried moving the code to the following standalone script, and I can reproduce > the error with that. In the original code w is kept as an instance variable, and > r is passed to asyncore.dispatcher.__init__ and probably kept there. Yes, the socket bound to `r` also gets bound to `self.socket` by this call: asyncore.dispatcher.__init__ (self, r) > I simulate that by returning them, then the caller of socktest can keep them > around. > > I try to call socktest from different processes A and B (two pythons): > (w,r = socktest()) > The call in A gets port 19999. The second call, in B, either blocks, or takes > over port 19999 (I see the second process taking over the port in a port scanner.) Sorry, I can't reproduce this -- but you didn't give a test program, just an isolated function, and I'm not sure what you did with it. I called that function in an infinite loop, appending the return value to a global list, with a short (< 0.1 second) sleep between iterations, and closed the returned sockets fifty iterations after they were created. Ran that loop in two processes. No hangs, or any other oddities, for some minutes. It did _eventually_ hang-- and both processes at the same time --with netstat showing more than 4000 sockets hanging around in TIME_WAIT state then. I assume I bashed into some internal Windows socket resource limit there, which Windows didn't handle gracefully. Attaching to the processes under the MSVC 6 debugger, they were hung inside the MS socket libraries. Repeated this several times (everything appeared to work fine until > 4000 sockets were sitting in TIME_WAIT, and then both processes hung at approximately the same time). Concretely: sofar = [] try: while 1: print '.', stuff = socktest() # calling your function sofar.append(stuff) time.sleep(random.random()/10) if len(sofar) == 50: tup = sofar.pop(0) w, r = tup msg = str(random.randrange(1000000)) w.send(msg) msg2 = r.recv(100) assert msg == msg2, (msg, msg2) for s in tup: s.close() except KeyboardInterrupt: for tup in sofar: for s in tup: s.close() Note that there's also a bit of code there to verify that the connected sockets can communicate correctly; the `assert` never triggered. You haven't said which versions of Windows or Python you're using. I was using XP Pro SP2 and Python 2.3.5. Don't know whether either matters. It was certainly the case when I ran it that your > print port statement needed to display ports less than 19999 at times, meaning that the > a.bind((host, port)) did raise an exception at times. It never printed a port number less than 19997 for me. Did you ever see it print a port number less than 19999? > a.bind in B does not raise socket.error: (10048, 'Address already in use') as > expected, when the server socket in A is closed, even though the port is used by > the client socket r in A. I'm not sure what that's saying, but could be it's an illusion. For example, >>> import socket >>> s = socket.socket() >>> s.bind(('localhost', 19999)) >>> s.listen(2) >>> a1 = socket.socket() >>> a2 = socket.socket() >>> a1.connect(('localhost', 19999)) >>> a2.connect(('localhost', 19999)) >>> b1 = s.accept() >>> b2 = s.accept() >>> b1[0].getsockname() ('127.0.0.1', 19999) >>> b2[0].getsockname() ('127.0.0.1', 19999) >>> That is, it's normal for the `r` in > r, addr = a.accept() to repeat port numbers across multiple `accept()` calls, and indeed to duplicate the port number from the `bind` call. This always confused me (from way back in my Unix days -- it's not "a Windows thing"), and maybe it's not what you're talking about anyway. > If I remove a.close(), and keep a around (by passing it to the caller), a.bind > works as expected - it raises socket.error: (10048, 'Address already in use'). As above, I'm seeing `bind` raise exceptions regardless. > But in the litterature on sockets, I read it should be okay to close the server > socket and keep using the client sockets. > > So, is this a possible bug in bind() ? Sure feels that way to me, and I'm not seeing it (or don't know how to provoke it). But I'm not a socket expert, and am not sure I've ever met anyone who truly was ;-) > I have tested the new code from Tim Peters, it apparently works, ports are given > out by windows. > But could the same problem with bind occur here, since a is closed (and garbage > collected) ? (far less chance for that since we do not specify port numbers, I > know). > > I tried getting a pair of sockets with Tim's code, and then trying to bind a > third socket to the same port as a/r. And I got the same problem as above. Here I'm not sure what "the same problem" means, as you've described more than one problem. Do you mean that you get a hang? Or that you see suspiciously repeated port numbers? Or ...? Seeing concrete code might help. Last question for now: have you seen a hang on more than one flavor of Windows? Thanks for digging into this! [and Sune's code] > import socket, errno > > class BindError(Exception): > pass > > > def socktest(): > """blabla > """ > > address = ('127.9.9.9', 19999) > > a = socket.socket (socket.AF_INET, socket.SOCK_STREAM) > w = socket.socket (socket.AF_INET, socket.SOCK_STREAM) > > # set TCP_NODELAY to true to avoid buffering > w.setsockopt(socket.IPPROTO_TCP, 1, 1) > > # tricky: get a pair of connected sockets > host='127.0.0.1' > port=19999 > > while 1: > print port > try: > a.bind((host, port)) > break > except: > if port <= 19950: > raise BindError, 'Cannot bind trigger!' > port=port - 1 > > a.listen (1) > w.setblocking (0) > try: > w.connect ((host, port)) > except: > pass > r, addr = a.accept() > a.close() > w.setblocking (1) > > #return (a, w, r) > return (w, r) > #return w

2 20

RE: [Zope] zserver blocking behavior with a slow request
by Jim Abramson 03 Aug '05

03 Aug '05

> > I'm finding that once a user requests a page which runs a > particularly slow sql (say up to 5 minutes), > any other > subsequent requests seem to take the hit as well, and return > very slowly. > Take a look at DadlockDebugger product - you'll see what is > happening with your Zope threads. > Thanks, this looks quite handy, I am going to give it a whirl (I assume that threadframe module dependency hasn't yet hosed anyone's Linux machine - funny how a disclaimer can inspire paranoia) > Isn't it better idea to execute long running query outside of > zope - external method or sth, and only check for results > even with simple page reload? We did something like this with > pdf generation and this worked as expected, but of course you > need some additional work to create such thing. > Yes I agree, in my case those SELECTs aren't *supposed* to be taking 5 minutes. That's a separate problem. However when such a problem bites I don't want all of my users to suffer, just the unlucky one who requested the doomed sql to be run. > > So is this blocking effect just expected behavior for zope? > We are using Zope and Oracle too, and even 10 threads... and > I didn't noticed something like this. What is your processor > doing then? What is the CPU(s?) load. The single CPU isn't loaded at all. But if DeadlockDebugger doesn't turn up any good clues I will do some investigating at the system level. Thanks for advice, Jim

1 0

PCNG 1.2.7 does exception when being added.
by Vlada Macek 03 Aug '05

03 Aug '05

I have the following configuration, both Zope and Plone are up to date to these branches: svn://svn.zope.org/repos/main/Zope/branches/Zope-2_8-branch https://svn.plone.org/svn/plone/bundles/2.1 Plone version: Plone 2.1-beta2 (svn/unreleased) Zope version: Zope (Zope 2.8.1-b1, python 2.3.5, linux2) Python 2.3.5 (#2, May 4 2005, 08:51:39) [GCC 3.3.5 (Debian 1:3.3.5-12 Browser(s) and version(s): Firefox 1.0.4 Operating system: Linux Debian 3.1 (Sarge) stable When I install PloneCollectorNG 1.2.5 from tar.gz and try to add the "pcng tracker" from Plone, the following error will always rise: Error Type BadRequest Error Value The id "plaintext_lexicon" is invalid - it is already in use. The full log item from the Plone Error Log is attached. Is it that PCNG 1.2.7 cannot be used with recent SVN Zope/Plone? If so, couldn't this be mentioned somewhere in the docs, please? Thanks in advance. -- \//\/\ (Sometimes credited as 1494 F8DD 6379 4CD7 E7E3 1FC9 D750 4243 1F05 9424.) Čas 2005/08/03 18:45:45.912 GMT+2 Jméno uživatele admin (admin) Adresa URL požadavku http://localhost:8080/HIS/sczescwerheerhdfb/createObject Typ výjimky BadRequest Hodnota výjimky The id "plaintext_lexicon" is invalid - it is already in use. Traceback (innermost last): * Module ZPublisher.Publish, line 113, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 40, in call_object * Module Products.CMFFormController.FSControllerPythonScript, line 105, in __call__ * Module Products.CMFFormController.Script, line 141, in __call__ * Module Products.CMFCore.FSPythonScript, line 108, in __call__ * Module Shared.DC.Scripts.Bindings, line 311, in __call__ * Module Shared.DC.Scripts.Bindings, line 348, in _bindAndExec * Module Products.CMFCore.FSPythonScript, line 163, in _exec * Module None, line 16, in createObject <FSControllerPythonScript at /HIS/createObject used for /HIS/sczescwerheerhdfb> Line 16 * Module Products.ATContentTypes.lib.constraintypes, line 274, in invokeFactory * Module Products.CMFCore.PortalFolder, line 408, in invokeFactory * Module Products.CMFCore.TypesTool, line 930, in constructContent * Module Products.CMFCore.TypesTool, line 340, in constructInstance * Module Products.CMFCore.TypesTool, line 571, in _constructInstance * Module Products.PloneCollectorNG.Collector, line 4, in addPloneCollectorNG * Module OFS.ObjectManager, line 293, in _setObject * Module Products.PloneCollectorNG.Collector, line 122, in manage_afterAdd * Module Products.PloneCollectorNG.Collector, line 146, in setup_tools * Module Products.PloneCollectorNG.Collector, line 162, in _setup_catalog * Module Products.BTreeFolder2.BTreeFolder2, line 431, in _setObject * Module Products.PloneCollectorNG.Collector, line 952, in manage_afterAdd * Module Products.CMFCore.CatalogTool, line 184, in _initIndexes * Module OFS.ObjectManager, line 265, in _setObject * Module OFS.ObjectManager, line 84, in checkValidId BadRequest: The id "plaintext_lexicon" is invalid - it is already in use. REQUEST form type_name 'PloneCollectorNG' cookies tree-s 'eJzTyCkw5NLIKTDiClZ3hANXW3WuAmOuRKCECUjWFEnWxzsQJGvGlQiSNkfR5wuSsQDKJCbqAQD1tBZ8' zmi_top_frame '' zmi_use_css '1' I18N_LANGUAGE 'cs' dtpref_cols '95%' __ac_name 'admin' wstyle '' dtpref_rows '25' photo_display 'large' _ZopeId '40204748A18SzYLK0d0' lazy items other zmi_top_frame '' tree-s 'eJzTyCkw5NLIKTDiClZ3hANXW3WuAmOuRKCECUjWFEnWxzsQJGvGlQiSNkfR5wuSsQDKJCbqAQD1tBZ8' URL3 'http://localhost:8080' URL2 'http://localhost:8080/HIS' AUTHENTICATION_PATH '' LANGUAGE 'cs' AUTHENTICATED_USER admin type_name 'PloneCollectorNG' SERVER_URL 'http://localhost:8080' traverse_subpath [] SESSION id: 11230875371247919349, token: 40204748A18SzYLK0d0, contents: [] dtpref_cols '95%' __ac_name 'admin' dtpref_rows '25' ACTUAL_URL 'http://localhost:8080/HIS/sczescwerheerhdfb/createObject' _ZopeId '40204748A18SzYLK0d0' URL 'http://localhost:8080/HIS/sczescwerheerhdfb/createObject' I18N_LANGUAGE 'cs' wstyle '' PUBLISHED <FSControllerPythonScript at /HIS/createObject used for /HIS/sczescwerheerhdfb> controller_state <Products.CMFFormController.ControllerState.ControllerState object at 0x4226364c> TraversalRequestNameStack [] LANGUAGE_TOOL <Products.PloneLanguageTool.LanguageTool.LanguageBinding instance at 0x4488c90c> zmi_use_css '1' BASE1 'http://localhost:8080' BASE2 'http://localhost:8080/HIS' BASE3 'http://localhost:8080/HIS/sczescwerheerhdfb' BASE4 'http://localhost:8080/HIS/sczescwerheerhdfb/createObject' URL1 'http://localhost:8080/HIS/sczescwerheerhdfb' photo_display 'large' URL0 http://localhost:8080/HIS/sczescwerheerhdfb/createObject URL1 http://localhost:8080/HIS/sczescwerheerhdfb URL2 http://localhost:8080/HIS URL3 http://localhost:8080 BASE0 http://localhost:8080 BASE1 http://localhost:8080 BASE2 http://localhost:8080/HIS BASE3 http://localhost:8080/HIS/sczescwerheerhdfb BASE4 http://localhost:8080/HIS/sczescwerheerhdfb/createObject environ HTTP_COOKIE '__ac_name="admin"; zmi_top_frame=""; dtpref_cols="95%"; dtpref_rows="25"; photo_display=large; zmi_use_css="1"; wstyle=; tree-s="eJzTyCkw5NLIKTDiClZ3hANXW3WuAmOuRKCECUjWFEnWxzsQJGvGlQiSNkfR5wuSsQDKJCbqAQD1tBZ8"; _ZopeId="40204748A18SzYLK0d0"; I18N_LANGUAGE="cs"' SERVER_SOFTWARE 'Zope/(Zope 2.8.1-b1, python 2.3.5, linux2) ZServer/1.1 Plone/Unknown' SCRIPT_NAME '' REQUEST_METHOD 'GET' HTTP_KEEP_ALIVE '300' SERVER_PROTOCOL 'HTTP/1.1' QUERY_STRING 'type_name=PloneCollectorNG' channel.creation_time 1123087523 CONNECTION_TYPE 'keep-alive' HTTP_ACCEPT_CHARSET 'ISO-8859-2,utf-8;q=0.7,*;q=0.7' HTTP_USER_AGENT 'Mozilla/5.0 (X11; U; Linux i686; cs-CZ; rv:1.7.8) Gecko/20050517 Firefox/1.0.4 (Debian package 1.0.4-2)' HTTP_REFERER 'http://localhost:8080/HIS/sczescwerheerhdfb' SERVER_NAME '0.0.0.0' REMOTE_ADDR '127.0.0.1' PATH_TRANSLATED '/HIS/sczescwerheerhdfb/createObject' SERVER_PORT '8080' HTTP_HOST 'localhost:8080' HTTP_ACCEPT 'text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5' GATEWAY_INTERFACE 'CGI/1.1' HTTP_ACCEPT_LANGUAGE 'cs,en-us;q=0.7,en;q=0.3' HTTP_ACCEPT_ENCODING 'gzip,deflate' PATH_INFO '/HIS/sczescwerheerhdfb/createObject'

4 3

RE: [Zope] Mount points
by Pascal Peregrina 03 Aug '05

03 Aug '05

Ok will do. Thanks -----Original Message----- From: zope-bounces(a)zope.org [mailto:zope-bounces@zope.org] On Behalf Of Dieter Maurer Sent: 03 August 2005 18:36 To: Pascal Peregrina Cc: zope(a)zope.org Subject: Re: [Zope] Mount points Pascal Peregrina wrote at 2005-8-2 19:39 +0100: >Is there a way to automount clientstorages ? >(I mean by just editing zope.conf and restarting the Zope instance) No >Currently I have to add a ZODB Mount Point object through ZMI... You can automate this mount point creation in a script of yourself... -- Dieter _______________________________________________ Zope maillist - Zope(a)zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) ********************************************************************** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept by MIMEsweeper for the presence of computer viruses. www.mimesweeper.com **********************************************************************

1 0

Mount points
by Pascal Peregrina 03 Aug '05

03 Aug '05

Hi, Is there a way to automount clientstorages ? (I mean by just editing zope.conf and restarting the Zope instance) Currently I have to add a ZODB Mount Point object through ZMI... Thanks (I use Zope 2.8.0 / python 2.3.5) Pascal ********************************************************************** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept by MIMEsweeper for the presence of computer viruses. www.mimesweeper.com **********************************************************************

2 1

test for the existence of a nodeValue
by Jon Bowlas 03 Aug '05

03 Aug '05

Hi All, I'm pretty new to python and zope but would like to know how I can adapt this script below to test to see if nodeValue actually contains a value and if it doesn't then I would like to return a default value I can check for using my zpt that references this script: if not attobject: return '' version = attobject.get_viewable() nodes = version.content.documentElement.getElementsByTagName(attname) if not nodes: return '' return nodes[0].childNodes[0].nodeValue Two parameters are passed to this script attobject, attname. 'attobject' is an object id and 'attname' is the childNode with the nodeValue I wish to return. Hope someone can help, and I hope this makes sense. Thanks Jon

2 1

zserver blocking behavior with a slow request
by Jim Abramson 03 Aug '05

03 Aug '05

On a production Zope (2.7.6) running behind apache and connected to an Oracle database, I'm finding that once a user requests a page which runs a particularly slow sql (say up to 5 minutes), any other subsequent requests seem to take the hit as well, and return very slowly. I have 8 zserver threads and a ZODB pool size of 10. I'm fairly certain I'm not maxing these out. So is this blocking effect just expected behavior for zope? One point of note is that the requests passed from apache all bear the remote ip of 127.0.0.1. Could zserver be throttling the incoming requests due to their identical REMOTE_IP (or some other apache configuration detail)? Is there anything else I should investigate? OH btw, I am using my own connection pool and managing db interface / transactions manually, so ZSQL should not be considered a culprit. Furthermore I'm watching the connections/pool themselves and they are not maxed either. Thanks for any help. Jim

4 3

When "Debug Spinning Zope" doesn't work
by Paul Winkler 03 Aug '05

03 Aug '05

It used to be that on the rare occasions that Zope hung, I could use the recipe at http://zopelabs.com/cookbook/1073504990 to diagnose the problem (usually my own code was at fault). But, what does it mean when I get nothing at all from "info threads" in gdb? I get this on two different systems: system A: python2.3.5 zope 2.7.7 gdb 5.2-2 system B: python 2.3.4 zope 2.7.3 gdb 5.2-2 Lately this seems to happen regardless of whether Zope is responsive or not. e.g.: $ cat Z2.pid $ cat /zope/ZopeInstanceHome/var/Z2.pid 24161 $ sudo gdb python2.3 Password: GNU gdb Red Hat Linux (5.2-2) Copyright 2002 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux"... (gdb) attach 24161 Attaching to program: /usr/local/bin/python2.3, process 24161 Reading symbols from /lib/libpthread.so.0...done. (snip copious output) Reading symbols from /usr/local/lib/python2.3/lib-dynload/itertools.so...done. Loaded symbols for /usr/local/lib/python2.3/lib-dynload/itertools.so 0x080775af in PyDict_GetItem (op=0x44d16934, key=0x401ae020) at Objects/dictobject.c:498 498 return (mp->ma_lookup)(mp, key, hash)->me_value; (gdb) info threads {gdb) That's it, nothing. It used to work for me. Any clues? -- Paul Winkler http://www.slinkp.com

3 3

RE: [Zope] How to instruct my Zope to use another pytrhon than the oneit was budled with.
by Jim Abramson 03 Aug '05

03 Aug '05

> > I also have another Python version on my system and i was > wondering how could i "tell" Zope to use that one instead of > the one it comes with from the installation executable package. > If you start up using the runzope.bat script in ZOPE_HOME/bin, just edit the value of PYTHON to point to the desired alternate executable. Don't automatically expect things to work smoothly on the first try - watch the output from startup and see if you need to pull missing dependencies into the sys.path (edit the value of PYTHONPATH if this is necessary) - or if the executable is even compatible at all (version, distribution/standard library, etc). Jim

1 0