[Zope] Backing Up Zope (was: Re: [Zope] Data.fs.lock?)

Thu, 29 Jun 2000 17:24:22 -0400

This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_001_01BFE210.654C999A
Content-Type: text/plain;
	charset="iso-8859-1"

> -----Original Message-----
> From: Erik Enge [mailto:erik@src.no]
> 
> On Thu, 29 Jun 2000, Jeff K. Hoffman wrote:
> 
> > It is my understanding that the ZODB on disk is always in a 
> consistent
> > state; Jim has said, in the past, that you can simply make a copy of
> > Data.fs, even while Zope is running, without problems.
> 
> But the Data.fs has to be updated sometimes.  And if you read 
> from a file
> that is changing, the integrity of your data isn't.  How can 
> you make sure
> that the file is not written to during a certain period of time?  You
> can't?

You can.

This would be consistent with "atomic updates."  What it means is
that a transaction is not written to the Data.fs file until it has
completed, and when it is finally written out, it's done as an
atomic operation, i.e. at the system level it is a single write
operation and nothing else happens on the system until it completes.

From that you can infer that if you were reading the Data.fs file
(making your backup copy) at the same time, it would be impossible
to read the file "in the middle of an update."  If both read and
write requests hit Data.fs at essentially the same time, one goes
first and the other waits until the first completes.  So if the
write gets first shot, the read will block until the write (meaning
a completed transaction/update) finishes, and your read will include
that last update.  If your read gets first shot, then it will not
include the update.  You will under no circumstances get *half* an
update.

One of the other things that makes this possible is ZODB being of
a "journaling" nature.  i.e., all updates are *appended* to Data.fs.
This is why you must periodically pack the database, which
conglomerates all the updates back into the base datastore (or
whatever those wacky Zopistas call it. ;) that comprises the first
part of the Data.fs file.

I think that's all correct.  Someone please throttle me senseless
if not.  :-)

-cw-

------_=_NextPart_001_01BFE210.654C999A
Content-Type: text/html;
	charset="iso-8859-1"

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="Generator" CONTENT="MS Exchange Server version 5.5.2650.12">
<TITLE>RE: [Zope] Backing Up Zope (was: Re: [Zope] Data.fs.lock?)</TITLE>
</HEAD>
<BODY>

&gt; -----Original Message-----
 &gt; From: Erik Enge [<A HREF="mailto:erik@src.no">mailto:erik@src.no</A>]
 &gt; 
 &gt; On Thu, 29 Jun 2000, Jeff K. Hoffman wrote:
 &gt; 
 &gt; &gt; It is my understanding that the ZODB on disk is always in a 
 &gt; consistent
 &gt; &gt; state; Jim has said, in the past, that you can simply make a copy of
 &gt; &gt; Data.fs, even while Zope is running, without problems.
 &gt; 
 &gt; But the Data.fs has to be updated sometimes.&nbsp; And if you read 
 &gt; from a file
 &gt; that is changing, the integrity of your data isn't.&nbsp; How can 
 &gt; you make sure
 &gt; that the file is not written to during a certain period of time?&nbsp; You
 &gt; can't?

<P><FONT SIZE=2>You can.</FONT>
</P>

This would be consistent with &quot;atomic updates.&quot;&nbsp; What it means is
 that a transaction is not written to the Data.fs file until it has
 completed, and when it is finally written out, it's done as an
 atomic operation, i.e. at the system level it is a single write
 operation and nothing else happens on the system until it completes.

<P><FONT SIZE=2>From that you can infer that if you were reading the Data.fs file</FONT>
<BR><FONT SIZE=2>(making your backup copy) at the same time, it would be impossible</FONT>
<BR><FONT SIZE=2>to read the file &quot;in the middle of an update.&quot;&nbsp; If both read and</FONT>
<BR><FONT SIZE=2>write requests hit Data.fs at essentially the same time, one goes</FONT>
<BR><FONT SIZE=2>first and the other waits until the first completes.&nbsp; So if the</FONT>
<BR><FONT SIZE=2>write gets first shot, the read will block until the write (meaning</FONT>
<BR><FONT SIZE=2>a completed transaction/update) finishes, and your read will include</FONT>
<BR><FONT SIZE=2>that last update.&nbsp; If your read gets first shot, then it will not</FONT>
<BR><FONT SIZE=2>include the update.&nbsp; You will under no circumstances get *half* an</FONT>
<BR><FONT SIZE=2>update.</FONT>
</P>

One of the other things that makes this possible is ZODB being of
 a &quot;journaling&quot; nature.&nbsp; i.e., all updates are *appended* to Data.fs.
 This is why you must periodically pack the database, which
 conglomerates all the updates back into the base datastore (or
 whatever those wacky Zopistas call it. ;) that comprises the first
 part of the Data.fs file.

<P><FONT SIZE=2>I think that's all correct.&nbsp; Someone please throttle me senseless</FONT>
<BR><FONT SIZE=2>if not.&nbsp; :-)</FONT>
</P>

<P><FONT SIZE=2>-cw-</FONT>
</P>

</BODY>
</HTML>
------_=_NextPart_001_01BFE210.654C999A--