[Andy Yates] ...
Let's not go into the gory details. Another discussion of the complexities of memory management will not be productive.
I agree.
The fact remains that if given enough time Zope will use so much memory that it crashes the server. I've reproduced this on at least 3 Linux servers with a clean install of Zope. The pattern SEEMS to be the more data I put in a session object the faster the server dies. This can not be normal behavior. I'm used to servers running for months. I have to restart Zope every 3 days at least. I need to at least have a long weekend away form this thing! ;-)
Can anybody please verify my results. I've given a very simple test case. If I'm only one in the world that can cause this to happen then fine. At least that narrows my search to something internal to our system.
I think people are trying to help, and you need to follow up more. For example, I didn't see a reply to Tres's questions in this thread, and I didn't see what looked like a particularly relevant reply to Dieter's observation that the test setup appears to provoke an unbounded number of sessions each with a 45-minute timeout (although Dieter appeared to miss this part of the message he was replying to: The session timeout does not seem to affect the memory use. We even tried setting the session timeout to 1 minute. your reply to him didn't have any indication of how many requests/sessions were getting created; and you didn't respond to his point that he limits max sessions to 5000 -- Tres also asked about max # of sessions). ...
Please don't take offence I'm sure the BTree code is really cool.
Oh, no offense at all! C code usually leaks like a sieve when it's new <wink>. I was just trying to steer you away from the BTree code in this case, because it's not new, has benefited from a lot of work to plug leaks, and just isn't a high-probability suspect anymore. BTW, you *do* see BTree building blocks from the debug control panel; they're in the lines like: BTrees.OOBTree.OOBTree 2019 11997 +9978 BTrees.OOBTree.OOBucket 1297 5512 +4215
I'm just grasping at straws. When this problem started I was told the problem must be in MY code. So I removed ALL my code and made a very simple test case that reproduces the problem. Then I was told well you just don't understand how memory REALLY works.
I didn't see anyone say that in this thread. Maybe I missed it.
At this point I don't really care how memory really works. I just want to be able to put data in session objects without my server crashing.
Sure.
Run my test case and see what happens. I double dog dare you! ;-)
Sorry, but I don't have time to dig into this at all. My replies have just regurgitated info I knew off the top of my head. That doesn't mean I think your problem is trivial, it just means I can't volunteer substantive help.