[Zope] ZEO hogging too much memory

[Wankyu Choi]

I forgot to post a followup to this thread. ( I did post a followup to =
the
same thread on the DirectoryStroage maillinglist ;-)

I solved this problem by applying the aa1 patch to the 2.4.20 kernel.

If any of you experience a similar problem ( kswapd hogging the =
resources
and never letting it go, etc ) try the aa1 patch. I also tried gentoo
patches, but they didn't fix the problem.

The problem was: when your low memory is scarce, kswapd kicks in and =
hogs
all the remaining resources. A simple packing attempt of more than 1G
storage would reproduce this madness with a vanilla 2.4.20 kernel. Even =
when
things get cool, kswapd wouldn't let go the resources it's hogging. The
cached memory, for example, is there for you to see but none of your
processes can't retrieve any of it. The faster your IO devices (I've got =
a
hardware raid, for example), the worse this problem gets.

Try the following patch:

=09
http://www.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.20a=
a1.
bz2

My problems gone with this patch.

Cheers,
Wankyu Choi
---------------------------------------------------------------
  Wankyu Choi
  CEO/President
  NeoQuest Communications, Inc.
  http://www.zoper.net
  http://www.neoboard.net
---------------------------------------------------------------  =20


-----Original Message-----
From: Oliver Bleutgen [mailto:myzope@gmx.net]=20
Sent: Wednesday, April 16, 2003 9:02 PM
To: Wankyu Choi
Cc: zope@zope.org
Subject: Re: [Zope] ZEO hogging too much memory


Wankyu Choi wrote:
> I just stopped the ZEO server and found all that 2G memory it was=20
> using didn't free up at all.
>=20
> I'm running ZEO 2.01b1
>=20
> Leaking?

I'm not an expert at linux kernel internals, but AFAIK even if the=20
server is leaking, the kernel should reclaim the space when the process=20
is stopped.
You wrote you use kernel 2.4.0, that's very brave I'd say. I know it's=20
easier said than done, but you better first install an newer linux=20
kernel before looking at the zope stuff. You get at least the promise of =

a saner out-of-memory behavior.
Other things you can do are setting process limits, so that the ZEO/zope =

servers cannot bring down the whole machine.
There's also a zope specific product from chrism, autolance (search=20
zope.org) which makes zope monitoring and restarting itself, when some=20
conditions (memory etc.) are met.


HTH,
oliver