[Zope-dev] DB.close() needs to be called

[Barry A. Warsaw]

>>>>> "TD" == Toby Dickenson <tdickenson@geminidataloggers.com> writes:

    >> worse" part.  If you've enable autopacking and you don't
    >> cleanly close the storage, you won't exit the process because
    >> the autopack thread won't get stopped and joined.

    TD> Yes, I had the same problem with DirectoryStorage, which uses
    TD> a seperate thread to move writes from its journal directory
    TD> into the database directory.

Ah.  Is this transactional?  What would happen if the thread got
killed in the middle of the move?

    >> We could make the
    >> autopack thread a daemon process

    TD> Thats how DirectoryStorage now works.

Hmm, maybe we make it a daemon thread and put a timeout on the join,
so we'll try to exit cleanly and won't hang if we can't.  Autopacking
should be safe transactionally, but we'd have to do a recovery if it
got killed uncleanly.

    >> It wouldn't be good, but remember that we run BerkeleyDB in
    >> auto-recovery mode, so it means that the next time you open the
    >> database, it will run recovery (you can also run recovery
    >> manually).  However, depending on your checkpointing policy,
    >> and the size of committed data since your last checkpoint,
    >> recovery could end up taking a long time.

    TD> Last time I looked at your BerkeleyDB storages, the
    TD> administrator needed to implement his own checkpointing
    TD> policy. I always thought that was a disadvantage. Would it now
    TD> be a good idea for the storages to trigger their own
    TD> checkpointing inside the autopacker thread?

Actually, now you can configure the storages to automatically
checkpoint every nth ZODB transaction.  Checkpointing in a thread may
or may not provide additional benefit.

    >> Sure, any number of bad things can happen at any time, and
    >> defense against that is one of the benefits of using BerkeleyDB
    >> underneath.  But under normal operations, we definitely want to
    >> exit cleanly.

    TD> Agreed, on both points.

Cool.
-Barry