Re: [Zope-CMF] Using XRon to backup my portal
Just a quick question about it: What happens to the ongoing transactions? I'm using W2K and from other large-file copies I've done it looks like the original file is frozen to changes, unless it's completely uploaded in memory. Do I need to stop Zope in order to allow the OS to perform the copy? Thanks in advance, Ausum ----- Original Message ----- From: "Jens Vagelpohl" <jens@zope.com> To: "Jeff Ross" <thejeffross@hotmail.com> Cc: "zope-cmf" <zope-cmf@zope.org> Sent: Sunday, October 27, 2002 12:53 AM Subject: Re: [Zope-CMF] Using XRon to backup my portal
just making regularly scheduled copies of your entire Data.fs file would be so much simpler...
jens
i have never seen it that the original file is "frozen" during that copy operation. i have successfully copied Data.fs files up to 5 GB (on linux) without any indication that the website had any trouble. if you can verify that the original turns "read-only" during the copy then that could be a windoze-only thing. jens On Sunday, Oct 27, 2002, at 01:05 US/Eastern, Ausum Studio wrote:
Just a quick question about it:
What happens to the ongoing transactions? I'm using W2K and from other large-file copies I've done it looks like the original file is frozen to changes, unless it's completely uploaded in memory.
Do I need to stop Zope in order to allow the OS to perform the copy?
Thanks in advance,
Can anyone confirm it? I'd like to have a theoretical understanding of this subject besides the practical one, because I suspect that there might come up byte-level inconsistencies at the copied file, not the original one, when a file larger than the memory allocated for the application using it is being copied at the same it's been appended. I base my case on the idea that, as data need to be copied by chunks, there would exist the possibility that not already copied data could have changed during the copy of the first chunks, meaning that the final copied file would be different to the original one (at the time the copy started), and not exactly equal to the same file at the time the copy ended. Thanks in advance, Ausum ----- Original Message ----- From: "Jens Vagelpohl" <jens@zope.com> To: "Ausum Studio" <ausum_studio@hotmail.com>
i have never seen it that the original file is "frozen" during that copy operation. i have successfully copied Data.fs files up to 5 GB (on linux) without any indication that the website had any trouble. if you can verify that the original turns "read-only" during the copy then that could be a windoze-only thing.
jens
On Sunday, Oct 27, 2002, at 01:05 US/Eastern, Ausum Studio wrote:
Just a quick question about it:
What happens to the ongoing transactions? I'm using W2K and from other large-file copies I've done it looks like the original file is frozen to changes, unless it's completely uploaded in memory.
Do I need to stop Zope in order to allow the OS to perform the copy?
Thanks in advance,
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
FileStorage only appends at the end of the file. the only thing that could happen is bad records at the end of your copy. the ZODB code is already intelligent enough to shave that off all by itself. why don't you just try and see how it works? jens On Sunday, Oct 27, 2002, at 19:51 US/Eastern, Ausum Studio wrote:
Can anyone confirm it?
I'd like to have a theoretical understanding of this subject besides the practical one, because I suspect that there might come up byte-level inconsistencies at the copied file, not the original one, when a file larger than the memory allocated for the application using it is being copied at the same it's been appended.
I base my case on the idea that, as data need to be copied by chunks, there would exist the possibility that not already copied data could have changed during the copy of the first chunks, meaning that the final copied file would be different to the original one (at the time the copy started), and not exactly equal to the same file at the time the copy ended.
Thanks in advance,
Ausum
----- Original Message ----- From: "Jens Vagelpohl" <jens@zope.com> To: "Ausum Studio" <ausum_studio@hotmail.com>
i have never seen it that the original file is "frozen" during that copy operation. i have successfully copied Data.fs files up to 5 GB (on linux) without any indication that the website had any trouble. if you can verify that the original turns "read-only" during the copy then that could be a windoze-only thing.
jens
On Sunday, Oct 27, 2002, at 01:05 US/Eastern, Ausum Studio wrote:
Just a quick question about it:
What happens to the ongoing transactions? I'm using W2K and from other large-file copies I've done it looks like the original file is frozen to changes, unless it's completely uploaded in memory.
Do I need to stop Zope in order to allow the OS to perform the copy?
Thanks in advance,
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
On Monday 28 October 2002 1:41 am, Jens Vagelpohl wrote:
FileStorage only appends at the end of the file.
Not entirely true. FileStorage still supports a non-transactional undo mechanism that writes bytes to the middle of files. This mechanism is not normally used by Zope, however it might be used by other non-Zope ZODB applications, or custom products. (could it be exploited by an attacker who wanted to break your backups? hmmmm) Proviing that this type of live backup is safe requires knowledge about how the backup program will read the file. The obvious approach of reading from start to end is compatible with FileStorages append-only approach, but not all backup programs operate that way. I prefer to take a copy of the data.fs using 'cp' (which I know to be safe), and backup that.
why don't you just try and see how it works?
Thats a fantasically dumb approach for testing a backup strategy. Just because you cant see any problems on a couple of test runs, it doesnt mean: 1. there arent any hidden problems. 2. there wont be obvious problems when you come to rely on your backup.
Toby> I prefer to take a copy of the data.fs using 'cp' (which I know to Toby> be safe), and backup that. How do you know 'cp' is safe? Just because it copies front to back? I think most backup programs can recognize if a file changes as they copy it and refuse to back it up in that situation. -- Skip Montanaro - skip@pobox.com http://www.mojam.com/ http://www.musi-cal.com/
participants (4)
-
Ausum Studio -
Jens Vagelpohl -
Skip Montanaro -
Toby Dickenson