problem with Zeo, it hangs a lot and never reply the request
Hi!! I was using Zope with zeo, two client machines, connecetd to a main machine. I was running this for about a month, and it was working OK. For some reason, that i dont know, stop working. I didn't change any configuration in any of the computers, the problem started a week and a half ago, but these monday the Zeo crash. This is the problem: If i start zope with ZEO, when a client request a page the ZeoClient hangs a lot and never reply the request. What i meant is that the browser mantain an Status of "opening www.mydomain.com" (on IE)but never display the page, and never tell me any error. In Nestacpe sent me a "Time Out Request", and in Lynx stays in a status of "Waiting for reply". Fir0st of all i thought that may be was a communication problem, but I use Lynx locally and does the same thing. Secondly i thougth it may be was the Squid cache, but here i my reason to think that is ZEO: If start Zope without ZEO (No ZSS and no Client servers, only one machine running Zope and point all my squid caches to the server running zope) the site works fine, display the pages fine. And if I start the ZSS and the CS in the machines the site does the problem above mentioned. I use this command to start ZEO (the same config that appears in the Zope Book) python lib/python/ZEO/start.py -p 8800 -h 192.168.0.1 My clients run with the custom_zodb.py with 192.168.0.1 as the host and 7700 as the server port. The clients listen on the 8080 port. Just for the record i have running Client servers in all the machines (the ones with the squid and the one with the ZSS running) I have cheked teh stupid_log_file, the Z2.log and the one that zope creates with -M parameter, and neither of them sent me some error or tell me that something is wrong. I tried to use the configuration that Kedai shows in the zope site (http://www.zope.org/Members/kedai/UseZeoZope), but i run in some problems when i try to start the clients This the output of the ZEO_EVENTS.log(using the kedai method) host zope]# tail -f var/ZEO_EVENTS.log ------ 2002-08-07T04:09:05 INFO(0) zdaemon zdaemon: Tue Aug 6 23:09:05 2002: Houston, we have forked ------ 2002-08-07T04:09:05 INFO(0) zdaemon zdaemon: Tue Aug 6 23:09:05 2002: Hi, I just forked off a kid: 25027 ------ 2002-08-07T04:09:05 INFO(0) zdaemon zdaemon: Tue Aug 6 23:09:05 2002: Houston, we have forked ------ 2002-08-07T04:09:10 INFO(0) ZEO Server Serving main: <ZODB.FileStorage.FileStorage instance at 0x83085d4> ------ 2002-08-07T04:09:10 INFO(0) ZEO Server Listening on ('192.168.0.1', 7700) The ZEO server starts aparently fine and when i start the client this what it sents me to the output zope]# Unhandled exception in thread: Traceback (most recent call last): File "/usr/local/zope/lib/python/ZEO/zrpc.py", line 94, in connect self.aq_parent.notifyConnected(s) File "/usr/local/zope/lib/python/ZEO/ClientStorage.py", line 143, in notifyConnected LOG("ClientStorage", INFO, "Connected to storage") File "/usr/local/zope/lib/python/zLOG/__init__.py", line 130, in LOG log_write(subsystem, severity, summary, detail, error) TypeError: object of type 'None' is not callable Hope some one can help me. I couldnt tested with the Kedai configuration because of the problem of the client... And in the form that i read in the Zope Book has the hangs. Thx Tavo
Gustavo M. Palomares Reyes writes:
... python lib/python/ZEO/start.py -p 8800 -h 192.168.0.1
My clients run with the custom_zodb.py with 192.168.0.1 as the host and 7700 as the server port. The clients listen on the 8080 port. As you write: ZEO waits on port 8800 but clients want to connect at port 7700. A typo or a no-go?
.... LOG("ClientStorage", INFO, "Connected to storage") File "/usr/local/zope/lib/python/zLOG/__init__.py", line 130, in LOG log_write(subsystem, severity, summary, detail, error) TypeError: object of type 'None' is not callable "log_write" appears not be initialized correctly. Apparently, it is "None".
Enable Zope logging by setting the environment variable "STUPID_LOG_FILE" to your log file (you can do that with an argument to the Zope startup script). And see, whether the problem goes away. Dieter
----- Original Message ----- From: "Dieter Maurer" <dieter@handshake.de> To: "Gustavo M. Palomares Reyes" <gpalomares@proceso.com.mx> Cc: <zope@zope.org> Sent: 07 August, 2002 12:39 Subject: Re: [Zope] problem with Zeo, it hangs a lot and never reply the request
Gustavo M. Palomares Reyes writes:
... python lib/python/ZEO/start.py -p 8800 -h 192.168.0.1
My clients run with the custom_zodb.py with 192.168.0.1 as the host and 7700 as the server port. The clients listen on the 8080 port. As you write: ZEO waits on port 8800 but clients want to connect at port 7700. A typo or a no-go?
Sorry it was a typo. I mean the same port as the ZSServer the clients run also on the 8800 Tavo
.... LOG("ClientStorage", INFO, "Connected to storage") File "/usr/local/zope/lib/python/zLOG/__init__.py", line 130, in LOG log_write(subsystem, severity, summary, detail, error) TypeError: object of type 'None' is not callable "log_write" appears not be initialized correctly. Apparently, it is "None".
Enable Zope logging by setting the environment variable "STUPID_LOG_FILE" to your log file (you can do that with an argument to the Zope startup script). And see, whether the problem goes away.
Ok Dieter , I did waht you told me, I put the Stupid_log _File variable, and now not only the client Crashes also de ZEO Server. I did two things 1) I changed the permisions in my var/ directory so it could only be accessed (rwx) by my zope user but this the output of the logs ZEO_EVENTS: 2002-08-07T20:07:31 INFO(0) ZEO Server Serving main: <ZODB.FileStorage.FileStorage instance at 0x830a60c> ------ 2002-08-07T20:07:31 INFO(0) ZEO Server Listening on ('192.168.0.1', 8800) ------ 2002-08-07T20:08:09 INFO(0) ZEO Server Connect 136372896 ('192.168.0.1', 50364) ------ 2002-08-07T20:08:09 ERROR(200) ZEO Server Unknown storage_id: 1 ------ 2002-08-07T20:08:09 INFO(0) ZEO Server Close 136372896 ------ 2002-08-07T20:08:09 ERROR(200) ZEO uncaptured python exception, closing channel <ZEO.StorageServer.ZEOConnection connected 192.168.0.1:50364 at 0x820e2a0> (exceptions.TypeError:len() of unsized object [/usr/local/Zope-2.5.1-src/ZServer/medusa/asyncore.py|poll|94] [/usr/local/Zope-2.5.1-src/ZServer/medusa/asyncore.py|handle_read_event|391] [/usr/local/Zope/lib/python/ZEO/smac.py|handle_read|101] [./lib/python/ZEO/StorageServer.py|message_input|220] [./lib/python/ZEO/StorageServer.py|get_info|276]) ------ 2002-08-07T20:08:09 INFO(0) ZEO Server Close 136372896 And this the output when i tried to start the client Unhandled exception in thread: Traceback (most recent call last): File "/usr/local/Zope/lib/python/ZEO/zrpc.py", line 94, in connect self.aq_parent.notifyConnected(s) File "/usr/local/Zope/lib/python/ZEO/ClientStorage.py", line 143, in notifyConnected LOG("ClientStorage", INFO, "Connected to storage") TypeError: object of type 'None' is not callable --------------------------------------- 0 -------------------------------------------------------- 2) if a leave the permissions the way there where since setup this are the outputs: *ZEO_EVENTS 2002-08-07T20:33:55 INFO(0) ZEO Server Serving main: <ZODB.FileStorage.FileStorage instance at 0x830a60c> ------ 2002-08-07T20:33:55 INFO(0) ZEO Server Listening on ('172.16.39.84', 8800) ------ 2002-08-07T20:34:11 INFO(0) ZEO Server Connect 136365528 ('172.16.39.84', 57943) ------ 2002-08-07T20:34:11 ERROR(200) ZEO Server Unknown storage_id: 1 ------ 2002-08-07T20:34:11 INFO(0) ZEO Server Close 136365528 ------ 2002-08-07T20:34:11 ERROR(200) ZEO uncaptured python exception, closing channel <ZEO.StorageServer.ZEOConnection connected 172.16.39.84:57943 at 0x820c5d8> (exceptions.TypeError:len() of unsized object [/usr/local/Zope-2.5.1-src/ZServer/medusa/asyncore.py|poll|94] [/usr/local/Zope-2.5.1-src/ZServer/medusa/asyncore.py|handle_read_event|391] [/usr/local/Zope/lib/python/ZEO/smac.py|handle_read|101] [./lib/python/ZEO/StorageServer.py|message_input|220] [./lib/python/ZEO/StorageServer.py|get_info|276]) ------ 2002-08-07T20:34:11 INFO(0) ZEO Server Close 136365528 *Output when I start the client: Unhandled exception in thread: Traceback (most recent call last): File "/usr/local/Zope/lib/python/ZEO/zrpc.py", line 70, in connect 'Trying to connect to server: %s' % `connection`) TypeError: object of type 'None' is not callable *Output of the stupid_log_file: ----- 2002-08-07T20:34:11 INFO(0) client Trying to connect to server: ('172.16.39.84', 8800) ------ 2002-08-07T20:34:11 INFO(0) ClientStorage Connected to storage ------ 2002-08-07T20:34:11 PROBLEM(100) ClientStorage Disconnected from storage ------ 2002-08-07T20:34:11 PANIC(300) z2 Startup exception Traceback (innermost last): File /usr/local/Zope/z2.py, line 497, in ? File <string>, line 1, in ? File /usr/local/Zope-2.5.1-src/lib/python/Zope/__init__.py, line 45, in ? File /usr/local/Zope-2.5.1-src/lib/python/ZODB/DB.py, line 77, in __init__ File /usr/local/Zope/lib/python/ZEO/ClientStorage.py, line 294, in load (Object: ('172.16.39.84', 8800)) File /usr/local/Zope/lib/python/ZEO/zrpc.py, line 168, in __call__ TypeError: exceptions must be strings, classes, or instances, not None ------ 2002-08-07T20:34:11 PROBLEM(100) zdaemon zdaemon: Wed Aug 7 15:34:11 2002: The kid, 22002, died on me. The permissions are set to rwx----- to Data.fs and Data.fs.in to zope user rw-r--r-- to all the rest of Data.fs.* , stupid log, ZEO_EVENTS, ZEO_SERVER.pid, Z2.log under zope user rw-r--r-- to zProces.... Also i notice that a file name c1-8800-0.zec whit rw-r--r-- under root was created, Inow this is because of the ZEO_CLIENT=8800 enviroment variable in the start script. what can be wrong? I use the custom_zodb.py import ZEO.ClientStorage Storage=ZEO.ClientStorage.ClientStorage(('192.168.0.1',8800)) and the StorageConfig.py as described in te documentation of kedai (http://www.zope.org/Members/kedai/UseZeoZope) Im using Zope 2.5.1 and ZEO 1.0 over Redhat 7.2 and python 2.1.3 Tavo
Gustavo M. Palomares Reyes writes:
... ZEO server and ZEO client die ... File "/usr/local/Zope/lib/python/ZEO/ClientStorage.py", line 143, in notifyConnected LOG("ClientStorage", INFO, "Connected to storage") TypeError: object of type 'None' is not callable Sad, "LOG" is still "None".
I fear, you will need some debugging and check why "LOG" is 'None'. The problem is detected in "ClientStorage.py, line 143". Start there. Check, where "LOG" is imported from. See, how it is supposed to be initialized. Check, why it is not. You can use "print" statements or use the "pdb" debugger (which is not very nice). More advanced debuggers come, e.g., with WingIDE (but this costs something). Dieter
Ok, I have in mi ZEO_start script this ###zeo_start #!/bin/sh reldir=`dirname $0` exec python $reldir/lib/python/ZEO/start.py -S main=StorageConfig:main_storage -p 8800 -h 10.10.10.2\ ZEO_SERVER_PID=$reldir/var/ZEO_SERVER.pid \ STUPID_LOG_FILE=$reldir/var/ZEO_EVENTS.log \ "$@" my StorageConfig.py : import ZODB.FileStorage main_storage=ZODB.FileStorage.FileStorage('/usr/local/Zope/var/Data.fs') And when I kept out the -S main=StorageConfig:main_storage all my clients starts ok. What is the difference running the zope with or without the -S parameter? My other question is if you know ZEO? Because i could start my two Clients, it seems like everithing works fine, but when I connect my squids (on both clients machines to accelerate the site) to those ZEOclients, the site hangs a lot, and some times never respond (the browser keeps thinking). And when i disconnected ZEO( my both clients and my Server) and only start Zope as usual (only one instance) and connect both Squids to that instance the site works fine. Do you have any Idea why this happens? A month a go everithing was workinf fine, last sunday the disck of my Main server (wher the Zserver is) filled up. So what i did ,was remove logs to free some sapce, but when I started the ZEO Server and the clients, the hangs started. So i thought may be some files where corrupted becuase of the disk full. So I made a new installation of Zope with new ZEO (both main server and client machines) thinking that this could solve the problem, but it didnt. Just for the record I copied all the Data.Fs and Data.fs.* of my old installation to the new installation. But the new installation didnt solve the problem. Is there any possibility that the Data.fs be corrupted because of the disck full? And if so why it works fine whit one Zope and wit Zeo dont (remebre I use squid to accelerate ZOPE)? Thx Tavo ----- Original Message ----- From: "Dieter Maurer" <dieter@handshake.de> To: "Gustavo M. Palomares Reyes" <gpalomares@proceso.com.mx> Cc: <zope@zope.org> Sent: 08 August, 2002 14:31 Subject: Re: [Zope] problem with Zeo, it hangs a lot and never reply the request
Gustavo M. Palomares Reyes writes:
... ZEO server and ZEO client die ... File "/usr/local/Zope/lib/python/ZEO/ClientStorage.py", line 143, in notifyConnected LOG("ClientStorage", INFO, "Connected to storage") TypeError: object of type 'None' is not callable Sad, "LOG" is still "None".
I fear, you will need some debugging and check why "LOG" is 'None'.
The problem is detected in "ClientStorage.py, line 143". Start there. Check, where "LOG" is imported from. See, how it is supposed to be initialized. Check, why it is not.
You can use "print" statements or use the "pdb" debugger (which is not very nice). More advanced debuggers come, e.g., with WingIDE (but this costs something).
Dieter
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Gustavo M. Palomares Reyes writes:
.... And when I kept out the -S main=StorageConfig:main_storage all my clients starts ok. What is the difference running the zope with or without the -S parameter? I do not know...
My other question is if you know ZEO? We use it without problems.
However, I would not say I were a ZEO expert...
.... Is there any possibility that the Data.fs be corrupted because of the disck full? Yes, but this should not cause big problems; especially, Zope looking at the "Data.fs" directly should have the same problems than ZEO looking at the same "Data.fs". You may run "fsrecover" on your "Data.fs" (in the ZODB package) and see whether it finds something strange.
And if so why it works fine whit one Zope and wit Zeo dont A mystery...
The problem might be some corrupted "pyc" file belonging to the ZEO server. We once had a mysteriously truncated "*.pyc" file leading to weird and difficult to explain errors. Maybe, you remove all "*.pyc" files and restart ZEO/Zope. The fact, that some variables contain "None" whereas they should contain a function, does not indicate a "Data.fs" problem but some code problem. Dieter
participants (2)
-
Dieter Maurer -
Gustavo M. Palomares Reyes