[ZODB-Dev] Maximum pool size considered harmful

[Tim Peters]

[Jim Fulton, sketches a scheme to eliminate the max pool size, which
 later got fleshed out at
    http://zope.org/Wikis/ZODB/SimplifyConnectionManagement
]
>> - Keep a weak-key dictionary (weak set?) of connections
>>   so we can do things like compute connection and
>>   cache statistics
>>
>> - Keep a stack of unused connections, for reuse
>>
>> - When someone calls open(), if the unused connection stack is
>>   non-empty, they'll get the connection from the top of the stack,
>>   otherwise they'll get a new connection.
>>
>> - When someone closes a connection, the connection will be
>>   added to the top of the stack. If the stack is over a maximum size,
>>   connections are discarded from the bottom of the stack (ie deck).

[Martin Kretschmar]
> but the implementation should be a little bit more
> thread aware.

Could you be more specific about what you mean, preferably phrased as a
modification to what Jim described?

> I once had hard to trace ODBC access violations under
> load and after 1-2 days. The problem was a recycled
> CDatabase object. After I gave every thread his "own"
> object back, the problem was gone.

What does this have to do with what Jim proposed?  It may be obvious, but I
don't see it (and neither did Jim <wink>).

> Creating a session to an Oracle data base can also be
> quite a pain, since it takes at least 5 seconds. So
> it is a good idea to be able to keep sessions open
> and at the same time being able to enforce their
> closure.

Similarly.  If you need to keep using a specific ZODB connection, don't
close it.  It *sounds* like you want the implementation to guess something
about your app, but I'm not sure what, or how it could be implemented.

The stack-like behavior Jim described for connection recycling is intended
to reuse ZODB caches effectively in common access patterns (like opening and
closing a connection per request), and that's all.