[ZODB-Dev] Backing up a ZODB

Tim Peters tim at zope.com
Wed Oct 29 18:18:44 EST 2003


[Tim Peters]
>> That little status byte is first set to 'c', then the new
>> transaction is appended, and then it seeks back to the location of
>> the status byte and overwrites it with a blank.  That's the way in
>> which filestorage isn't changed solely by appending.

[Neil Schemenauer]
> Interesting.  Is that done because the size of the transaction is
> not known before it is written?

We'd have to channel Jim to know why it was done this way to begin with.

> If it was, you could write the marker after the transaction.  That's how
> it worked in my head(obviously not corresponding to reality).  Ie.
>
>     <transaction size><transaction data><status byte>...

We know all the sizes in advance, but can't know whether the disk will run
out of space, or the power will go out (etc), when we actually do the
writes.  I expect Jim wanted a positive indication of failure in the file
(when failure obtained), and that almost all code traverses filestorages
from low offset to high, so that having the status byte at a low offset is
most efficient in that most-common case.

> We copy filestorages all the time.  It's a bummer that there is a
> small chance it will fail.  We are using repozo to backup the live
> database so I'm not too worried.

There's nothing dicey about copying a filestorage that isn't in active use;
the OP was asking about backing up a live database.  We copy .fs files all
the time too.  It's exceedingly rare to find a 'c' transaction in an offline
database, because if failure *does* occur while writing a 'c' transaction,
FileStorage does all that it can to truncate the .fs file (to chop off the
'c' transaction) before shutting down.  That can be frustrated by, say, a
power outage.  But a copy of an inactive .fs file with a 'c' transaction at
the end is in no worse shape than the original .fs file.  In a live
database, copying a 'c' transaction *can* create a copy in worse shap than
the original database.  repozo.py didn't get changed to avoid that case
because somebody out-thought it in advance of a disaster <wink> (as Anthony
said, older versions of repozo.py weren't as careful as the most recent
versions).




More information about the ZODB-Dev mailing list