MemoryError that brings my zope to its knees
In the last month I have started to get these errors using Zope 2.7.7 in a ZEO configuration with one client and one storage server: ------ 2005-09-12T23:46:07 ERROR(200) ZODB Couldn't load state for 0x055ba7 Traceback (most recent call last): File "/usr/local/Zope/lib/python/ZODB/Connection.py", line 600, in setstate self._set_ghost_state(obj, p) File "/usr/local/Zope/lib/python/ZODB/Connection.py", line 639, in _set_ghost_state state = unpickler.load() MemoryError ------ 2005-09-12T23:46:07 ERROR(200) SiteError Error while logging Traceback (most recent call last): File "/usr/local/Zope/lib/python/Products/SiteErrorLog/SiteErrorLog.py", line 142, in raising tb_text = ''.join( MemoryError Sometimes I see this too: ------ 2005-09-06T15:23:30 ERROR(200) ZServer uncaptured python exception, closing channel <select-trigger (pipe) at bd9104c> (exceptions.MemoryError: [/usr/local/lib/python2.3/asyncore.py|read|69] [/usr/local/lib/python2.3/asyncore.py|handle_read_event|390] [/usr/local/Zope/lib/python/ZServer/medusa/thread/select_trigger.py|handle_read|77] [/usr/local/lib/python2.3/asyncore.py|recv|342] [/usr/local/lib/python2.3/asyncore.py|recv|523]) And this one also brings the show down: ------ 2005-09-06T05:35:51 ERROR(200) zrpc:30832 can't decode message: "(M2'K\x00U\x06.reply(TW\x00\x01\x00((U\tOFS.Imageq\x01U..." ------ 2005-09-06T05:35:51 ERROR(200) zrpc-conn(C):localhost:6100 Error caught in asyncore Traceback (most recent call last): File "/usr/local/lib/python2.3/asyncore.py", line 69, in read obj.handle_read_event() File "/usr/local/lib/python2.3/asyncore.py", line 390, in handle_read_event self.handle_read() File "/usr/local/Zope/lib/python/ZEO/zrpc/smac.py", line 217, in handle_read self.message_input(msg) File "/usr/local/Zope/lib/python/ZEO/zrpc/connection.py", line 339, in message_input msgid, flags, name, args = self.marshal.decode(message) File "/usr/local/Zope/lib/python/ZEO/zrpc/marshal.py", line 39, in decode return unpickler.load() # msgid, flags, name, args MemoryError It completely locks up the zope until I restart. I added more RAM to the box and also bumped up the zeo cache-size to 100MB and 10000 objects in the zope.conf. I also changed over from binding to an external ethernet interface and bound the zeo stuff to the localhost interface. Any ideas on other things I could try? Thanks, Calvin -- S i x F e e t U p | "Nowhere to go but open-source" Silicon Valley: +1 (650) 401-8579 | Midwest: +1 (317) 861-5948 Toll-Free: 1-866-SIX-FEET mailto:calvin@sixfeetup.com http://www.sixfeetup.com | Zope Hosting from $19.95/month
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Calvin Hendryx-Parker wrote:
In the last month I have started to get these errors using Zope 2.7.7 in a ZEO configuration with one client and one storage server:
------ 2005-09-12T23:46:07 ERROR(200) ZODB Couldn't load state for 0x055ba7 Traceback (most recent call last): File "/usr/local/Zope/lib/python/ZODB/Connection.py", line 600, in setstate self._set_ghost_state(obj, p) File "/usr/local/Zope/lib/python/ZODB/Connection.py", line 639, in _set_ghost_state state = unpickler.load() MemoryError ------ 2005-09-12T23:46:07 ERROR(200) SiteError Error while logging Traceback (most recent call last): File "/usr/local/Zope/lib/python/Products/SiteErrorLog/SiteErrorLog.py", line 142, in raising tb_text = ''.join( MemoryError
Sometimes I see this too:
------ 2005-09-06T15:23:30 ERROR(200) ZServer uncaptured python exception, closing channel <select-trigger (pipe) at bd9104c> (exceptions.MemoryError: [/usr/local/lib/python2.3/asyncore.py|read|69] [/usr/local/lib/python2.3/asyncore.py|handle_read_event|390] [/usr/local/Zope/lib/python/ZServer/medusa/thread/select_trigger.py|handle_read|77] [/usr/local/lib/python2.3/asyncore.py|recv|342] [/usr/local/lib/python2.3/asyncore.py|recv|523])
That is not a problem -- some client disconnected before Zope finished processing the request.
And this one also brings the show down:
------ 2005-09-06T05:35:51 ERROR(200) zrpc:30832 can't decode message: "(M2'K\x00U\x06.reply(TW\x00\x01\x00((U\tOFS.Imageq\x01U..." ------ 2005-09-06T05:35:51 ERROR(200) zrpc-conn(C):localhost:6100 Error caught in asyncore Traceback (most recent call last): File "/usr/local/lib/python2.3/asyncore.py", line 69, in read obj.handle_read_event() File "/usr/local/lib/python2.3/asyncore.py", line 390, in handle_read_event self.handle_read() File "/usr/local/Zope/lib/python/ZEO/zrpc/smac.py", line 217, in handle_read self.message_input(msg) File "/usr/local/Zope/lib/python/ZEO/zrpc/connection.py", line 339, in message_input msgid, flags, name, args = self.marshal.decode(message) File "/usr/local/Zope/lib/python/ZEO/zrpc/marshal.py", line 39, in decode return unpickler.load() # msgid, flags, name, args MemoryError
It completely locks up the zope until I restart. I added more RAM to the box and also bumped up the zeo cache-size to 100MB and 10000 objects in the zope.conf. I also changed over from binding to an external ethernet interface and bound the zeo stuff to the localhost interface.
Any ideas on other things I could try?
- Try 'memtest' or an equivalent on the box to verify that you don't have a flaky RAM chip. - If you are RAM constrained, *reduce* the number of objects in the connection cache; more objects in that cache requires more RAM. Leaving the on-disk cache size high shouldn't be a problem. - Add swap space, if possible (swapping will be painful, but better than a lockup). Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFDJqbB+gerLs4ltQ4RAj4HAJ4vCNeuoqIH8kDXbVzp6YTbjDn2LgCgqnzo bNaH6hDz9QBA5gCg+xcCyCs= =osoi -----END PGP SIGNATURE-----
participants (2)
-
Calvin Hendryx-Parker -
Tres Seaver