On Fri, 2003-08-29 at 06:28, Chris Withers wrote:
Shane Hathaway wrote:
as a lower bound. (Note that the cache is still allowed to grow indefinitely within the scope of a request, however.)
Which is still pretty broken IMHO :-(
This is the single biggest cause of Zope becoming unresponsive for me: people doing silly things that drag hordes of objects into memory i na single request,
Now that would be an interesting feature: An upper bound on the number of objects a request is allowed touch, period. If a request requires more than that it's rolled back.
made worse by the fact that Python is extraordinarily bad at giving memory back to the OS after that request has been completed ;-)
On Linux (and probably other nixes), this is a libc (mis)feature. You see, malloc() is a libc abstraction. The memory the OS makes avaliable to each process is actually a contiguous block with a variable upper (or is it lower?) bound. This bound is usually changed thru the sbrk() call, which is a wrapper to the brk() call. The latter tells to the OS the amount of heap space the process think it needs. The former is an increment or decrement wrapper around the latter. When the process runs out of heap (malloc()) space, it (The libc in the python interpreter, in our case) calls sbrk() with a positive number (eg. it asks for more memory). Now, no matter how much memory is later free()d (or in our case, python objects deleted), the system never calls sbrk() with a negative value. If it were to do that, it (the libc) would first have to make sure that there are no pointers to allocated space still in the area. I'm not sure, but I think that the glibc in Linux just doesn't bother. Defragmentation of the area is not an option because, in C, due to pointer arithmetic, you can never be sure where all the pointers are, to change them to the new locations. Now this is all from memory (and a cursory read of the sbrk(2) man page) so the above could be full of, erm..., incorrections :-) Corrections to the above explanation are welcome. Cheers, Leo -- Ideas don't stay in some minds very long because they don't like solitary confinement.