[ZODB-Dev] Memory out of control when *NOT* changing objects

[Tim Peters]

[Dieter Maurer]
> ...
> This is unfortunately a problem with the current ZODB release. Hopefully,
> it will get better with future ones:
>
>   As you found out, your ZODB cache grows without limits.
>
>   Usually, the ZODB connection performs a cache
>   garbage collection at transaction boundaries.
>   However, it can do this only, when it is informed about
>   such boundaries.
>
>   In the current ZODB (ZODB 3.2, at least), a ZODB connection
>   is only informed about transaction boundaries when
>   at least one of its objects registered with the transaction
>   (because it was modified). If only reads are performed,
>   the connection is not notified.
>
>   This is a bug!

An exclamation point makes any statement preceding it true <wink>.

> For my MVCC implementation, a connection must learn about each
> transaction boundary whether or not one of its objects is modified.
> Therefore, I force the connection to register itself with the
> transaction. This way, it gets informed for every boundary.
>
> Hopefully, future ZODB connections will do something similar...

3.3 tries to be less persnickety, but it can be complicated.  Here's part of
3.3's Connection docstring:

    The logic for handling modifications assumes that the thread that
    opened a Connection (called db.open()) is the thread that will use
    the Connection.  If this is not true, you should pass synch=False
    to db.open().  When the synch option is disabled, some transaction
    boundaries will be missed by the Connection; in particular, if a
    transaction does not involve any modifications to objects loaded
    from the Connection and synch is disabled, the Connection will
    miss the transaction boundary.  Two examples of this behavior are
    db.undo() and read-only transactions.

The short course is that it should no longer be "a problem" for most uses,
but applications are still free to screw themselves, in multiple ways.

> You can work around this problem by calling "connection._incrgc()"
> yourself after a transaction boundary.

I think Connection.cacheGC() is preferred.  I think that mostly because
names with underscores "should be" treated as private in Python.  In this
specific case, I note that Connection._incrgc() doesn't even exist in ZODB
3.3, but Connection.cacheGC() (still) does.