[Zope] Zope backup
Toby Dickenson
tdickenson@geminidataloggers.com
Mon, 31 Mar 2003 14:24:38 +0100
On Sunday 30 March 2003 8:57 am, Dieter Maurer wrote:
> sean.upton@uniontrib.com wrote at 2003-3-28 14:18 -0800:
> > Copying a FileStorage while transactions are being appended to the end
> > of it potentially means that the copy completes with before the
> > transaction commit is totally flushed to the file, so there is always
> > the possibility that your backup is going to need to have half-written
> > transactions manually truncated to be used on either a replica or a new
> > Zope/ZSS instance; this isn't that big of a deal ....
>
> When I understand this right, then this is done automatically
> when the "FileStorage" is opened.
I think that is correct - Ive never needed this *manual* process after a
correct backup/restore cycle of FileStorage
This truncating process might be automatic, however it is slow because it
effectively needs to regenerate data.fs.index. It is impossible for a backup
of a live FileStorage to atomically backup this file.
The process of recreating the index needs to read the entire data.fs file into
memory from start to end. This process would be needed in between any
attempted incremental backup or incremental replication cycle. In comparison
DirectoryStorage is close to maximally efficient at incremental backups and
replications.
DirectoryStorage's efficiency with incremental operations comes at a cost; it
is slower than FileStorage to make or restore a full backup. Creating all
those tiny files in a filesystem is equivalent to recreating data.fs.index,
but slower.
(provided data.fs.index fits in memory of course. FileStorage is terminally
slow once you push it into swap.)
--
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson