RE: [Zope] Backing up Zope's database?
Martijn wrote:
I briefly saw the pack operation but I had no clue that it could be used
It isn't supposed to be used for backup purposes, just like Python wasn't written to be the world's best desktop calculator. It just turned out that way.
for backup purposes. Generally things that say 'remove' as in 'remove object revisions that are older than X days' don't associate with 'backup' in my mind.
Zope doesn't have a "make a backup file facility". Packing just happens to make a backup file. If what is really wanted is a "make a backup file facility", this would be an easy project for the community to submit some patches.
What does removing object revisions older than X days *mean*? Does that mean only previous revisions get removed, but the current state of the object is saved? I presume this is so, but my suggestion is
Correct.
we need some clear documentation on this, as right now it baffles me (of course this may be because I'm not a native speaker of English :).
Anyway, you could automate a pack that says something ridiculous, like pack anything older then one thousand days.
I saw some references to the Zope scheduler..would this be the thing that I'd use to automate backups?
There really isn't a Zope Scheduler facility, unfortunately. I'd expect it to become more prominent when concurrency arrives. Instead, use cron. Write a little ZRPC (or just httplib) script that connects to the Control Panel and executes a pack.
I see. So the hack would be to write an ineffective pack operation that runs every day (preferably before the external backup starts but not too long before it starts..hmm), so that Data.bbb.old is created (as a side effect).
Precisely.
Or possibly, none of the above will work and it's total hogwash. :^)
Before I start implementing the backup procedure I'd like some more washing of this hog, I think. An alternative strategy that occured to me (for Zope 1.10.2, I think..I haven't installed that one yet) would be to use file system export.
That's another option. I'm not sure how much the import/export machinery executes at the application level instead of the database level (where pack resides), so export *might* be a tad slower. Also it won't *really* be a full backup as it won't contain old revisions of objects.
It would also be nice if this backup could somehow be synchronized with the external tape backup procedure, so that the tape backup doesn't backup while Zope's still creating a backup, and such.
Uhh, I imagine that's one for you.
I'm sure I'm not the only one trying to make a proper backup of the Zope database. How is this done by other people?
We do backups all the time on Unix and have no problems. The Zope database format generally makes it pretty safe to do backups (as all write operations are appends). --Paul Paul Everitt Digital Creations paul@digicool.com 540.371.6909
Paul Everitt wrote:
Martijn wrote:
I briefly saw the pack operation but I had no clue that it could be used
It isn't supposed to be used for backup purposes, just like Python wasn't written to be the world's best desktop calculator. It just turned out that way.
Yeah, it's a side effect; I am sorry I wasn't clear in showing that I understood it wasn't meant to do this.
for backup purposes. Generally things that say 'remove' as in 'remove object revisions that are older than X days' don't associate with 'backup' in my mind.
Zope doesn't have a "make a backup file facility". Packing just happens to make a backup file. If what is really wanted is a "make a backup file facility", this would be an easy project for the community to submit some patches.
Right; since I think having a good backup is rather essential, I'll delve into the source code to see what pack does to create its backup, and then try to extend the Zope control panel with a backup feature. [snip]
I saw some references to the Zope scheduler..would this be the thing that I'd use to automate backups?
There really isn't a Zope Scheduler facility, unfortunately. I'd expect it to become more prominent when concurrency arrives.
So currently there's no 'cron' type system in Zope, I understand? What do you mean by 'when concurrency arrives' by the way? Does this have to do with multithreading?
Instead, use cron. Write a little ZRPC (or just httplib) script that connects to the Control Panel and executes a pack.
I should look into ZRPC. And on how to do cron on NT (sigh :). Perhaps arcserve can start a little script before it starts to do the backup. I don't know much about arcserve (I didn't install it), and I should delve further tomorrow in exactly *what* is failing. [file system export as another option]
That's another option. I'm not sure how much the import/export machinery executes at the application level instead of the database level (where pack resides), so export *might* be a tad slower. Also it won't *really* be a full backup as it won't contain old revisions of objects.
I see. I'll have to think about how important keeping old revisions is.
It would also be nice if this backup could somehow be synchronized with the external tape backup procedure, so that the tape backup doesn't backup while Zope's still creating a backup, and such.
Uhh, I imagine that's one for you.
The ficticious Zope scheduler could call some external method to check..whatever, yeah, that's up to me. :)
I'm sure I'm not the only one trying to make a proper backup of the Zope database. How is this done by other people?
We do backups all the time on Unix and have no problems. The Zope database format generally makes it pretty safe to do backups (as all write operations are appends).
So it's not a big problem to do a backup of a Zope database while it is in active use? I really do need to figure out why arcserve fails at backing up the database. Oddly enough just copying it in NT works just fine. Therefore, the simplest ad hoc solution (if arcserve allows this and it should otherwise it's braindead) is to create a small python script or batch file that copies the database, and then do a backup of this database. I don't expect this particular Zope installation to see very heavy use in the future anyway, but one never knows. I just want to be sure in this case; making a backup one isn't pretty sure about isn't worth making... Thanks for your replies! Martijn
Instead, use cron. Write a little ZRPC (or just httplib) script that connects to the Control Panel and executes a pack.
I should look into ZRPC. And on how to do cron on NT (sigh :). Perhaps arcserve can start a little script before it starts to do the backup. I don't know much about arcserve (I didn't install it), and I should delve further tomorrow in exactly *what* is failing.
NT has "at" that does the same thing as "cron". "Winat" gives it a GUI interface but is part of the resource kit. Phil. ----------------------------------------------------------- Philip Aylesworth mailto:purple@mnsi.net
Paul Everitt wrote:
Martijn wrote: Instead, use cron. Write a little ZRPC (or just httplib) script that connects to the Control Panel and executes a pack.
I should look into ZRPC. And on how to do cron on NT (sigh :). Perhaps arcserve can start a little script before it starts to do the backup. I don't know much about arcserve (I didn't install it), and I should delve further tomorrow in exactly *what* is failing.
Arcserve has options to run scripts prior to and after each backup/restore/copy session ;-) BTW, arcserve HATES doing backups of open files even when files are opened O_RDONLY. The arcserve job queue actually can be used as a nice scheduling system for servers when you leave the real job empty and only use before/after backup scripts. -- <- Ronald Offerman | ron@offerman.cx <- Root Powered Carrot Munchers Ltd. Inc. SA AG BV "This is Linux Country. On a quiet night, you can hear Windows NT reboot!" "Daddy, why do those people have to use Microsoft Windows?" "Don't stare, son; it's not polite." "M$ Windows NT, an accident waiting to happen" "What goes up, must come down. Ask any system administrator." ".sig too big? Flame me, I'm cold!"
Paul Everitt wrote:
Zope doesn't have a "make a backup file facility". Packing just happens to make a backup file. If what is really wanted is a "make a backup file facility", this would be an easy project for the community to submit some patches.
Actually, Zope was designed so that you could just copy the database, without shutting down Zope, to make a backup. Unfortunately, this was broken before Zope 1.10. Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (888) 344-4332 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
participants (5)
-
Jim Fulton -
Martijn Faassen -
Paul Everitt -
Philip Aylesworth -
ron＠rotflol.cx