[ZODB-Dev] ZEO client leaking memory?

[sean.upton@uniontrib.com]

Hmm... I have a similar problem, but I always assumed it was just a Catalog
BTree memory leak issue, not a ZEO issue.  I haven't seen anything weird in
my ZEO_EVENTS.log, but I run cron jobs at 4:00/4:30 AM to add between
500-5000 new items and then reindex a total group of about 10-15k objects.
At 5:00 AM, I have a cron job to restart Zope in order to deal with memory
leaks caused by this; if ZEO is at issue, I assume that adding more objects
might just result in something similar to what Chris is experiencing,
perhaps?  I still wonder if the memory leak issue is related to
Catalog/ZCatalog in Zope, and simply an application level-problem?

Sean

-----Original Message-----
From: Jeremy Hylton [mailto:jeremy@zope.com]
Sent: Thursday, October 04, 2001 10:06 AM
To: Chris Withers
Cc: ZODB Dev List
Subject: Re: [ZODB-Dev] ZEO client leaking memory?


>>>>> "CW" == Chris Withers <chrisw@nipltd.com> writes:

  CW> Hi, Just to let you guys know, I've noticed that my
  CW> lets-index-30,000-documents ZEO client appears to be leaking
  CW> memory. After doing about 10K docs, the ZEO _client_ process has
  CW> sucked so much memory that the machien churns to a painfully
  CW> slow statis...

We're seeing a very similar problem with an internal project that uses
ZEO.  The client-side consumes so much memory that the server side
fails with a MemoryError.  We haven't made much progress debugging,
because it has been possible to simply restart the job after a crash
and continue.  :-(.

It's hard to tell if this is a problem with ZEO or not.  If the client
is leaking memory, the leak could be in any of several places: in the
ClientStorage, in any of the Zope code used by the client, or in the
applications themselves.

The logging issues, incidentally, was for blather-level logging.  If
you crank the zLOG severity to its most verbose, ZEO will log every
message it sends.  Unfortunately, you pay some performance penalty for
this ability whether you enable that level of logging or not.  ZEO
must format strings and make zLOG calls, and it isn't until you
actually get into the zLOG implementation that you check the log level
and find out the message will get tossed.  Fortunately, we'ved wrapped
all these message-level log calls in "if __debug__:" tests.  If you
run python -O, you pay no penalty.  (The compiler knows statically
that __debug__ is false and doesn't generate any code for the if
statement.)

So python -O makes a lot of sense in a production ZEO environment.
The only thing you lose with python -O is the ability to run the
Python debugger in that interpreter.

  CW> This, of course, could be a myriad of different things. Anyone
  CW> got any clues on how I can find out what's going on?

[More on this later.]  Jim noted that a number of the cache related
calls take ages and do nothing when the age is 0.  They also work at
a granularity of three seconds, which means anything less than three
seconds is treated as zero.  I'll try to dig into that issue this
afternoon and have a better explanation.

Jeremy


_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://lists.zope.org/mailman/listinfo/zodb-dev