[ZODB-Dev] Backing up a ZODB
Tim Peters
tim at zope.com
Wed Oct 29 18:18:44 EST 2003
[Tim Peters]
>> That little status byte is first set to 'c', then the new
>> transaction is appended, and then it seeks back to the location of
>> the status byte and overwrites it with a blank. That's the way in
>> which filestorage isn't changed solely by appending.
[Neil Schemenauer]
> Interesting. Is that done because the size of the transaction is
> not known before it is written?
We'd have to channel Jim to know why it was done this way to begin with.
> If it was, you could write the marker after the transaction. That's how
> it worked in my head(obviously not corresponding to reality). Ie.
>
> <transaction size><transaction data><status byte>...
We know all the sizes in advance, but can't know whether the disk will run
out of space, or the power will go out (etc), when we actually do the
writes. I expect Jim wanted a positive indication of failure in the file
(when failure obtained), and that almost all code traverses filestorages
from low offset to high, so that having the status byte at a low offset is
most efficient in that most-common case.
> We copy filestorages all the time. It's a bummer that there is a
> small chance it will fail. We are using repozo to backup the live
> database so I'm not too worried.
There's nothing dicey about copying a filestorage that isn't in active use;
the OP was asking about backing up a live database. We copy .fs files all
the time too. It's exceedingly rare to find a 'c' transaction in an offline
database, because if failure *does* occur while writing a 'c' transaction,
FileStorage does all that it can to truncate the .fs file (to chop off the
'c' transaction) before shutting down. That can be frustrated by, say, a
power outage. But a copy of an inactive .fs file with a 'c' transaction at
the end is in no worse shape than the original .fs file. In a live
database, copying a 'c' transaction *can* create a copy in worse shap than
the original database. repozo.py didn't get changed to avoid that case
because somebody out-thought it in advance of a disaster <wink> (as Anthony
said, older versions of repozo.py weren't as careful as the most recent
versions).
More information about the ZODB-Dev
mailing list