Re: [Zope-dev] A thought for 2.6 about various leaks and memory problems
"Adrian Hungate" wrote moved to a different thread or earlier, before the fork, so that we could simply kill senile threads without killing the entire shooting match?
unfortunately apache does this with seperate processes, not threads. memory corruption and leaks aren't solved by offing threads, unfortunately. the equivalent would be to start a bunch of single threaded zeo clients on a machine, with a loadbalancer in front of them, and periodically off one. the only decent loadbalancers I've seen, however, are pretty heavy weight, flaky, limited in functionality, or some combination of all three. also, the start time for a new ZEO client is non-trivial. Anthony
Anthony Baxter wrote:
"Adrian Hungate" wrote
moved to a different thread or earlier, before the fork, so that we could simply kill senile threads without killing the entire shooting match?
unfortunately apache does this with seperate processes, not threads.
memory corruption and leaks aren't solved by offing threads, unfortunately.
the equivalent would be to start a bunch of single threaded zeo clients on a machine, with a loadbalancer in front of them, and periodically off one.
the only decent loadbalancers I've seen, however, are pretty heavy weight, flaky, limited in functionality, or some combination of all three.
also, the start time for a new ZEO client is non-trivial.
I actually think this is a great idea, but none of my friends here at ZC seem to agree. Nevertheless I've been collecting up some data and resources that should help anyone interested in moving this idea forward. First, you'll need a load balancer. Here is a lightweight one that sounds like a good match for Zope, although I haven't tried it: http://siag.nu/pen/ It actually tries to match clients with servers, which should lead to more ZODB cache hits, therefore lower memory consumption. Second, you'll need some ZEO clients. I have an independent ZODB application that relies on ZEO, and its start time is about 1 second, even though it accesses a 250 MB Data.fs. The only reason a Zope client takes so long is it reads and catalogs all the help files on startup! Figure out how to disable that silly code and startup time will be much lower (though not 1 second, since there's a lot of Python code to load). This arrangement would have a lot of benefits: - The global interpreter lock would no longer be an issue, so you'd be able to take better advantage of multiple processors; - A failed request would never affect other users; - You'd be able to use new Zope releases in production sooner, since "absolute" stability would be less of a requirement. One thing to consider is that Apache might be technically *less* stable than Zope. Most Apache modules are written in C and are not written by the core Apache hackers, providing a lot more opportunities for pointer errors than Zope does. But since Apache forks instead of using threads, it takes maximum advantage of the stability provided by the operating system. So as a whole, it is more stable. Can you imagine how stable Zope might be if it used a similar strategy? Shane
On Tue, 19 Mar 2002 11:02:27 -0500, Shane Hathaway <shane@zope.com> wrote:
First, you'll need a load balancer. Here is a lightweight one that sounds like a good match for Zope, although I haven't tried it:
It actually tries to match clients with servers, which should lead to more ZODB cache hits, therefore lower memory consumption.
Interesting. One complication with that scheme is that ther is no convenient place to put caching. If it goes between Pen and Zope, you need many instances of your cache. If it goes between Pen and the clients, Pen's client-clustering is defeated because it sees only one client. Ive been using squid with ICP to match *datasets* to servers. http://www.zope.org/Members/htrd/icp/intro Toby Dickenson tdickenson@geminidataloggers.com
Interesting. One complication with that scheme is that ther is no convenient place to put caching. If it goes between Pen and Zope, you need many instances of your cache. If it goes between Pen and the clients, Pen's client-clustering is defeated because it sees only one client.
Postgresql takes a similar (multi process) approach to achieving stability as apache. see pg 4 of http://developer.postgresql.org/pdf/transactions.pdf Think shared memory caching behind the Zope worker processes ;-) And web server / proxy shared memory caching in front of the Zope worker processes. Of course, for stability, there is also the possibility of doing a OpenBSD style audit of the entire Zope (and possibly python) code base to make the code as close to perfect (and therefore stable) as the OpenBSD OS is ;-) Volunteers? Adam
On Tue, Mar 19, 2002 at 12:07:12PM -0500, Adam Manock wrote:
Interesting. One complication with that scheme is that ther is no convenient place to put caching. If it goes between Pen and Zope, you need many instances of your cache. If it goes between Pen and the clients, Pen's client-clustering is defeated because it sees only one client.
Postgresql takes a similar (multi process) approach to achieving stability as apache.
As a long-time user and hanger-on of the PostgreSQL community, let me take umberage at that statement. While it's true that postgresql uses a multiprocess model, and that model does protect other backends from poorly written C extensions, any inexplicable backend crashes are treated as serious bugs by the pgsql core developers, tracked down and exterminated. So, while the robustness of the system is enhanced by being multiprocess, its stability is due to a rabid devotion to quality code (both design and implementation), and bug hunting.
see pg 4 of http://developer.postgresql.org/pdf/transactions.pdf
Think shared memory caching behind the Zope worker processes ;-)
And web server / proxy shared memory caching in front of the Zope worker processes.
Interestingly, a discussion about caching strategies for web client apps has been going on in the postgresql HACKERS mailing list. There's signifcant reluctance to putting it in the DB backend, because it's seen as the clients job to cache.
Of course, for stability, there is also the possibility of doing a OpenBSD style audit of the entire Zope (and possibly python) code base to make the code as close to perfect (and therefore stable) as the OpenBSD OS is ;-) Volunteers?
Yeah, that's what I just described above, I think, for the pgsql core. So it uses _both_. Ross P.S. note the PostgreSQL, postgresql, and pgsql are used interchangably to refer to the entire RDBMS system and project. The first is 'official' the last two are due to various levels of typing lazyness.
participants (5)
-
Adam Manock -
Anthony Baxter -
Ross J. Reedstrom -
Shane Hathaway -
Toby Dickenson