[ZODB-Dev] Re: [Zope-dev] DB.close() needs to be called

[Barry A. Warsaw]

>>>>> "TD" == Toby Dickenson <tdickenson@geminidataloggers.com> writes:

    >> What would happen if the thread got killed in the middle of the
    >> move?

    TD> A more interesting question is what happens if the thread is
    TD> killed when it has moved some, but not all, of the files which
    TD> relate to one transaction.

That's more what I was thinking about.
    
    TD> In DirectoryStorage terminology this leaves the data directory
    TD> in a state which "is not a snapshot". The only disadvantage is
    TD> that you can not use the tools which assume the data directory
    TD> is a snapshot: the checking tool, the incremental back tool,
    TD> and the replication tool.

Interesting.  Berkeley storage has something similar.  Let's say the
storage crashes in the middle of tpc_vote() or tpc_finish().  First,
BerkeleyDB's recovery should clean up any of its corrupt transactions.
Then the storage has its own recovery process for the "current"
transaction, i.e. the one we were in the middle of at the time of the
crash.  Depending on when the crash occurred, the storages will either
commit or abort the current transaction (if the crash happened in
tpc_finish(), we'll complete the commit, otherwise we abort the
transaction).

So I believe -- although this isn't battle tested -- that the Berkeley
storages should be pretty safe against inconsistency and corruption.

    >> We could make the autopack thread a daemon process
    >> TD> Thats how DirectoryStorage now works.  Hmm, maybe we make
    >> it a daemon thread and put a timeout on the join, so we'll try
    >> to exit cleanly and won't hang if we can't.

    TD> Because some of the operations performed in the daemon thread
    TD> take a long time to complete?

Potentially yes, although the steps in the pack process are BerkeleyDB
transactionally protected.  I think the mark-and-sweep phases are
examples of things that could take a long time.  It should be possible
to craft some escape hatches to do a more controlled shutdown.
    
    TD> Would it be possible to break those operations into
    TD> transactionally-coherent chunks which complete in a reasonable
    TD> time? close() could wait for the current chunk to finish.

Yep.

    >> Autopacking should be safe transactionally, but we'd have to do
    >> a recovery if it got killed uncleanly.

    TD> Doesnt having a good checkpointing strategy mean that this
    TD> should never take long?

I think checkpointing itself is a trade-off between the length of time
it takes to checkpoint and the length of time it could potentially
take to recover.

    >> TD> Last time I looked at your BerkeleyDB storages, the TD>
    >> administrator needed to implement his own checkpointing TD>
    >> policy. I always thought that was a disadvantage. Would it now
    >> TD> be a good idea for the storages to trigger their own TD>
    >> checkpointing inside the autopacker thread?
    >> Actually, now you can configure the storages to automatically
    >> checkpoint every nth ZODB transaction.  Checkpointing in a
    >> thread may or may not provide additional benefit.

    TD> Interesting. BerkeleyStorage's automatic checkpointing is
    TD> currently triggered inside a commit or abort. This means the
    TD> checkpoint overhead is incurred at the one time you dont want
    TD> it - while a user is waiting for his transaction to
    TD> commit. DirectoryStorage's main use for the thread is to
    TD> perform its equivalent asynchronously. Assuming your storage
    TD> is not saturated with writes (which only happens in benchmarks
    TD> ;-) then checkpointing happens for free.

That's a very interesting idea!  Hmm, checkpoint in a thread.  Yeah,
that sounds much better, although I'm mildly concerned with
concurrancy issues, especially given that we don't use the BerkeleyDB
locking subsystem.  I'll have to read up on the docs, but it's a good
idea.

-Barry