On Tue, May 25, 2004 at 06:40:01AM +0100, Ben Last (Zope) wrote:
[mailto:zope-bounces+zope=benlast.com@zope.org]On Behalf Of Paul Winkler Zope only appends to Data.fs. (It has been mentioned that it is *possible* to twiddle stuff in the middle of a filestorage, but the core Zope code is not supposed to do this, and I've never heard an example of any add-on product that does so either.)
Out of curiosity, what then prevents Data.fs growing without bound? Does it get truncated from the beginning?
No, never. Instead, we "pack" it - a process which deletes transactions older than a given date. I've never really looked into how packing is implemented for the default storage, but I know that it makes a backup copy (Data.fs.old), tells Zope to write new transactions since the pack started to Data.fs.tmp, and finally when done with its work appends Data.fs.tmp to the new Data.fs. If you never pack, your Data.fs will indeed only ever get larger... OTOH, be sure you do your homework and know what version of Zope you have and whether it has known pack bugs. I know that 2.6.1-2.6.2 had pack bugs and could lose data during a pack. To the best of my knowledge, 2.6.4 and 2.7.0 do not have this bug.
Even if it does only get appended to, I assume there's a race condition where cp may encounter Data.fs with half or a record written to it and thus copy an inconsistent file.
Not a problem. Due to the ZODB's transactional nature, all this means is that the last transaction in the backup is incomplete and invalid and Zope will ignore it. If we number transactions with consecutive integers, then a Data.fs with complete transactions 0..N and incomplete transaction N+1 is functionally equivalent to a Data.fs that ends with complete transaction N. -- Paul Winkler http://www.slinkp.com