ZODB packaging behaviour: bug or feature?
Use case: Create a new Zope instance. Perform some changes and then pack the database to remove objects revisions older to 0 days. You want a clean instance to start with. Now create a cron job to daily pack the database to remove objects older to 14 days, and start to use your instance. Zope won't pack the database at all, and I guess it will be for the first 2 weeks, because it won't find revisions older to 14 days during that time. Is that normal behaviour? Telling by the interface text at the Database Management tab, it looks like the command would create a Data.fs.old file even though there weren't older revisions than the number set, meaning that by default the packing would create an exact copy of the current Data.fs (to keep all changes), at least until there were revisions to remove. The problem is that if you rely on Data.fs.old for backup purposes, you'll find that your 13th day Data.fs.old is exactly as that of day one. An even though you had never packed the database to 0 days, the problem would come up if, for any reason, you feel like it's time to manually pack the database to 0, in between those two weeks. Your cron job won't modify the existing Data.fs.old for the following two weeks. Am I missing something? Thanks in advance, Ausum
[Ausum Studio]
Use case: Create a new Zope instance. Perform some changes and then pack the database to remove objects revisions older to 0 days. You want a clean instance to start with. Now create a cron job to daily pack the database to remove objects older to 14 days, and start to use your instance. Zope won't pack the database at all, and I guess it will be for the first 2 weeks, because it won't find revisions older to 14 days during that time.
Is that normal behaviour?
I don't know what "normal" means, but it's the behavior you're asking for in that scenario.
Telling by the interface text at the Database Management tab, it looks like the command would create a Data.fs.old file even though there weren't older revisions than the number set, meaning that by default the packing would create an exact copy of the current Data.fs (to keep all changes), at least until there were revisions to remove.
If you're using a current ZODB (anything at or after 3.2.1c1; 3.2.5 is the latest stable release in the 3.2 line), an attempt to pack to a time earlier than the most recent time ever packed to shouldn't do anything more than raise a FileStorageError with this detail: The database has already been packed to a later time or no changes have been made since the last pack
The problem is that if you rely on Data.fs.old for backup purposes,
Sorry, that's not a reasonable backup strategy. The intended way to do FileStorage backup is explained here: http://zope.org/Wikis/ZODB/FileStorageBackup
you'll find that your 13th day Data.fs.old is exactly as that of day one.
Which is one reason for doing backups the intended way <wink>.
An even though you had never packed the database to 0 days, the problem would come up if, for any reason, you feel like it's time to manually pack the database to 0, in between those two weeks.
Manually packing the database to 0 is a novel meaning for "never packed the database to 0 days" <wink>.
Your cron job won't modify the existing Data.fs.old for the following two weeks.
Am I missing something? Thanks in advance,
You're missing that regular backups are needed regardless of packing -- tying backups to packing is poor practice. The intended backup procedure is independent of when pack does or doesn't happen to get rid of something. In fact, it goes faster if packing doesn't occur (so long as that's true, it can do quick incremental backups instead of a full backup -- read the page referenced above).
Tim Peters wrote:
If you're using a current ZODB (anything at or after 3.2.1c1; 3.2.5 is the latest stable release in the 3.2 line), an attempt to pack to a time earlier than the most recent time ever packed to shouldn't do anything more than raise a FileStorageError with this detail:
The database has already been packed to a later time or no changes have been made since the last pack
I'm sure I remember discussions where it was agreed that this annoying behaviour would be dropped? (or at least turned into a subclass that could be easily caught, rather than having to catch FileStorageError and then do a string comparison on the exception arguments :-( )
Manually packing the database to 0 is a novel meaning for "never packed the database to 0 days" <wink>.
Why not? (sorry for being slow) cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
[Tim Peters]
If you're using a current ZODB (anything at or after 3.2.1c1; 3.2.5 is the latest stable release in the 3.2 line), an attempt to pack to a time earlier than the most recent time ever packed to shouldn't do anything more than raise a FileStorageError with this detail:
The database has already been packed to a later time or no changes have been made since the last pack
[Chris Withers]
I'm sure I remember discussions where it was agreed that this annoying behaviour would be dropped? (or at least turned into a subclass that could be easily caught, rathe than having to catch FileStorageError and then do a string comparison o the exception arguments :-( )
It was changed (to log a message instead). I see that it's in ZODB 3.3 (under svn), but not on the Zope-2_7-branch branch of ZODB3 (under CVS). Don't know what happened there. Looks like we're missing several changes that were checked in -- maybe they're only on ZODB3 HEAD? Something's screwed up here indeed.
Manually packing the database to 0 is a novel meaning for "never packed the database to 0 days" <wink>.
Why not? (sorry for being slow)
No idea what you're asking here.
[Tim Peters]
If you're using a current ZODB (anything at or after 3.2.1c1; 3.2.5 is the latest stable release in the 3.2 line), an attempt to pack to a time earlier than the most recent time ever packed to shouldn't do anything more than raise a FileStorageError with this detail:
The database has already been packed to a later time or no changes have been made since the last pack
[Chris Withers]
I'm sure I remember discussions where it was agreed that this annoying behaviour would be dropped? (or at least turned into a subclass that could be easily caught, rathe than having to catch FileStorageError and then do a string comparison o the exception arguments :-( )
[Tim]
It was changed (to log a message instead). I see that it's in ZODB 3.3 (under svn), but not on the Zope-2_7-branch branch of ZODB3 (under CVS). Don't know what happened there. Looks like we're missing several changes that were checked in -- maybe they're only on ZODB3 HEAD? Something's screwed up here indeed.
After more staring, same story: the changes are in ZODB 3.3 (used with Zope 2.8 and Zope 3), but not on CVS's Zope-2_7-branch branch of ZODB3. I don't intend to address anything other than critical bugfixes in the ZODB 3.2 line anymore, so I don't intend to change this. If someone else wants to merge in this specific change on Zope-2_7-branch, that would be OK by me.
Tim Peters wrote:
Manually packing the database to 0 is a novel meaning
for "never packed the database to 0 days" <wink>.
Why not? (sorry for being slow)
No idea what you're asking here.
I guess I didn't understand what you meant by:
Manually packing the database to 0 is a novel meaning for "never packed the database to 0 days" <wink>
After more staring, same story: the changes are in ZODB 3.3 (used with Zope 2.8 and Zope 3), but not on CVS's Zope-2_7-branch branch of ZODB3. I don't intend to address anything other than critical bugfixes in the ZODB 3.2 line anymore, so I don't intend to change this. If someone else wants to merge in this specific change on Zope-2_7-branch, that would be OK by me.
http://zope.org/Collectors/Zope/1668 I'll try and remember to grab it on a bug day. Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
participants (3)
-
Ausum Studio -
Chris Withers -
Tim Peters