RE: [Zope] Backing Up Zope (was: Re: [Zope] Data.fs.lock?)
Just to start by pointing out the bloody obvious: - Restoring from backup means you lose all data between backup date/time and system failure. Sucks, but it beats losing *all* your data. (RAID5 anyone?) - With that in mind, *consistency* of the backup database takes the highest priority. All existing transactions should be fully completed - no half-done updates, etc. It seems to me that the easiest way to get there with ZODB might be to just make a simple file copy of the database and have some kind of integrity checking tool to run against the static backup copy to weed out the junk. Is this possible, does it already exist? Probably a dumb question, but would Zope itself do this if you fired it off against an inconsistent ZODB? Another random thought is that if ZODB transactions and writes are atomic, than none of this should be an issue. Anyone know the answer to that one? -cw-
-----Original Message----- From: Chris Withers [mailto:chrisw@nipltd.com] Sent: Thursday, June 29, 2000 7:49 AM To: Erik Enge Cc: Oleg Broytmann; Zope Mailing List Subject: [Zope] Backing Up Zope (was: Re: [Zope] Data.fs.lock?)
Erik Enge wrote:
On Thu, 29 Jun 2000, Chris Withers wrote:
Hmm, about extending this so you have 'rotating data.fs
files' in the
same way you have rotating log files?
In general, yes :-). Do you think it would be solving the right problem the right way?
I think so, but that's only my view.
If anyone's got any better ideas, please pipe up :-)
cheers,
Chris
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
On Thu, 29 Jun 2000, Wilkinson Charlie E wrote:
Another random thought is that if ZODB transactions and writes are atomic, than none of this should be an issue. Anyone know the answer to that one?
It is my understanding that the ZODB on disk is always in a consistent state; Jim has said, in the past, that you can simply make a copy of Data.fs, even while Zope is running, without problems. Based on that, I think the answer to the above is yes.
-cw-
--Jeff --- Jeff K. Hoffman 704.849.0731 x108 Chief Technology Officer mailto:jeff.hoffman@goingv.com Going Virtual, L.L.C. http://www.goingv.com/
On Thu, 29 Jun 2000, Jeff K. Hoffman wrote:
It is my understanding that the ZODB on disk is always in a consistent state; Jim has said, in the past, that you can simply make a copy of Data.fs, even while Zope is running, without problems.
But the Data.fs has to be updated sometimes. And if you read from a file that is changing, the integrity of your data isn't. How can you make sure that the file is not written to during a certain period of time? You can't?
On Thu, 29 Jun 2000, Erik Enge wrote:
On Thu, 29 Jun 2000, Jeff K. Hoffman wrote:
It is my understanding that the ZODB on disk is always in a consistent state; Jim has said, in the past, that you can simply make a copy of Data.fs, even while Zope is running, without problems.
But the Data.fs has to be updated sometimes. And if you read from a file that is changing, the integrity of your data isn't. How can you make sure that the file is not written to during a certain period of time? You can't?
Now that I'm in the spotlight I can't seem to find the message I was referring to in the archives; I sure hope I didn't misquote anyone (i.e. Jim). Unfortunately, I don't have an answer to your question. I still believe I am correct, but cannot tell you, technically, why. I have some theories, but I don't want to cloud the issue anymore with my hypotheses. Let's hope someone who does know chimes in. --Jeff --- Jeff K. Hoffman 704.849.0731 x108 Chief Technology Officer mailto:jeff.hoffman@goingv.com Going Virtual, L.L.C. http://www.goingv.com/
"Jeff K. Hoffman" wrote:
On Thu, 29 Jun 2000, Erik Enge wrote:
On Thu, 29 Jun 2000, Jeff K. Hoffman wrote:
It is my understanding that the ZODB on disk is always in a consistent state; Jim has said, in the past, that you can simply make a copy of Data.fs, even while Zope is running, without problems.
But the Data.fs has to be updated sometimes. And if you read from a file that is changing, the integrity of your data isn't. How can you make sure that the file is not written to during a certain period of time? You can't?
Now that I'm in the spotlight I can't seem to find the message I was referring to in the archives; I sure hope I didn't misquote anyone (i.e. Jim).
Unfortunately, I don't have an answer to your question. I still believe I am correct, but cannot tell you, technically, why. I have some theories, but I don't want to cloud the issue anymore with my hypotheses.
Let's hope someone who does know chimes in.
Quite simply, Data.fs is appended, not modified, except during a pack operation. Think of it like a binary log file. Therefore there are only two ways a simple, live backup of Data.fs can fail: 1) You do the copy in the middle of a pack operation. For most, packing is a rare operation. But if this happened, your backup would be worthless. 2) You get a partly truncated record at the very end. This is actually not a big deal, and AFAIK in most cases ZODB is capable of automatically rolling back a partially committed transaction. Shane
On Thu, 29 Jun 2000 17:19:32 -0400, Shane Hathaway <shane@digicool.com> wrote:
Quite simply, Data.fs is appended, not modified, except during a pack operation. Think of it like a binary log file. Therefore there are only two ways a simple, live backup of Data.fs can fail:
1) You do the copy in the middle of a pack operation. For most, packing is a rare operation. But if this happened, your backup would be worthless.
I think its better than that. FileStorage packs itself into a temporary file, which is renamed to data.fs only when the pack is complete.
2) You get a partly truncated record at the very end. This is actually not a big deal, and AFAIK in most cases ZODB is capable of automatically rolling back a partially committed transaction.
I think the other exception is during an Undo, where Zope will rewrite one byte in the middle of the file. Toby Dickenson tdickenson@geminidataloggers.com
Wilkinson Charlie E wrote:
Just to start by pointing out the bloody obvious:
- Restoring from backup means you lose all data between backup date/time and system failure. Sucks, but it beats losing *all* your data. (RAID5 anyone?)
A) Please: no html! B) I just had one weird and wild thought. A RAID5 type of storage using ZODBs instead of blocl devices. Probably not doable, and might not gain anything if doable, but it was one of those ideas that shoots across your brain and sounds cool at first. Just thought I'd share. :) -- "Linux: the operating system with a CLUE... Command Line User Environment". seen in a posting on comp.software.testing
participants (6)
-
Bill Anderson -
Erik Enge -
Jeff K. Hoffman -
Shane Hathaway -
Toby Dickenson -
Wilkinson Charlie E