[ZODB-Dev] Maximum pool size considered harmful

[Jim Fulton]

Does anyone like the current strategy for limiting the number of
database connections?

I don't. :)

When I implemented this strategy, I was afraid that people would create
too many connections and exhast system resources.  The thing is, at least
in Zope, the number of connections is limited by the number of threads, except
when there is a programming error causing connections to leak.  If connections
leak (fail to get closed and returned to the pool), then after a while,
the application locks up because threads can't get any more connections.
While this stragety does prevent system resources from being exhausted through
excesssive connection creation, I think the end result isn't any better. ;)

I propose to radically simplify the connection management strategy of ZODB.DB.

I suggest that we:

- Keep a weak-key dictionary (weak set?) of connections
   so we can do things like compute connection and
   cache statistics

- Keep a stack of unused connections, for reuse

- When someone calls open(), if the unused connection stack is
   non-empty, they'll get the connection from the top of the stack,
   otherwise they'll get a new connection.

- When someone closes a connection, the connection will be
   added to the top of the stack. If the stack is over a maximum size,
   connections are discarded from the bottom of the stack (ie deck).

Thoughts?

Objections?

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org