Scott Robertson wrote:
I just posted this in the Collector but I wanted to get some feedback from the list as well. That way those greedy bastards at DC won't get all the fun of fixing and extending zope.
Oh, we're far too tired to be greedy. :^) Funny, getting jacked by your ISP (as we discussed on the phone today) and having to spend four weeks dialing our LAN into the net over a modem gives you a new...perspective. All I can say is, thank goodness for CodeIt getting the zope.org machine online so fast. (Does anybody else notice how much faster the Zope site connectivity is now? Pretty sweet...)
We Need some sort of method to keep memory usage in check. Currently a brand new instance with an empty database will consume 4 MB of
Hey, that's less than Netscape running on Linux! :^)
memory. If you upload 1 or 2 6 MB files into this database the running process swells up to 18 MB in memory and no matter how many times you
Hmm, though as you know I'm a rube, I beg to differ. Since I believe everything I hear, I can repeat the following with a straight face. In fact, I'm almost ashamed to send this email, thus publicly demonstrating my complete stupidity on the subject. But hey, it's only email. Python apps like Zope allocate memory then release memory. What the operating system does with the released memory is its problem. Most Unices seem to keep the virtual size at the highest point allocated by a process. That is, the number will won't automatically go down (though the real size will). If the operating system decides that something else needs the memory, it will take the memory back. NT, however, will do so. Or perhaps I've got it all wrong. Anyways, we had a major, major app -- the classified ad system that spawned Principia and then Zope -- where the customer insisted on running each newspaper in its own process. Then complained when their memory footprint swelled to gigs. We *easily* spent 300 manhours trying to do something here (actually, this is the genesis of the tunable object cache).
flush the cache the process still remains the same size. For that matter we're unable to decrease the number of objects stored in the cache as well. Maybe we are using it wrong?
It would be nice to be able to set a quota of 10 MB to the total amount of memory each process uses. Or to have more aggressive garbage collection.
Some requirements questions -- how do you want the policy to be implemented? a. Never let the process size go above the number. b. Check before each request to see if the process size is above the number. c. Check after. When the process size is exceeded by the first byte, what would you like to do?
If we can get a fix on this issue then running multiple instances of Zope from the same box should be a breeze. (Although maybe expensive, but I've always wanted to have a box with 1GB of RAM)
Are you running each of them with different Pythons? If you run them with the same Python, will they share the code segment thingy for the interpreter? What is the RSS in the above example, or in the case where you have four Zopes running? --Paul