[Zope] The Goblins of Zope

[Toby Dickenson]

On Tuesday 14 January 2003 6:32 pm, Dieter Maurer wrote:

>  > How does anyone know for sure that there is no problem in their stor=
ages
>  > today
>  > without using a "fsck" tool?
>
> You are much deeper in "Storages" than I am.
>
> I have the feeling that a "FileStorage" is a linear sequence
> of transaction records. When Zope builds the "FileStorage" index,
> I expect (I never verified) that is analyses the linear sequence
> and checks that at least the record sizes are correct

When building the index FileStorage scans through these headers as fast a=
s=20
possible. This will detect some damage but not all. Of course this check =
is=20
only performed when the index is built - on startup after an unclean=20
shutdown. During a clean shutdown the old index is persisted in=20
data.fs.index. If you always shutdown cleanly then this may never happen!

> Besides this elementary structure, there are backpointers from
> newer to older versions.

Yes, plus other redundant information.=20

For FileStorage, all of this is thoroughly tested with the fstest.py scri=
pt.=20
IMO it is prudent to run fstest at least as often as you pack, perhaps on=
=20
Data.fs.old.

> Damage inside a transaction record may not be detected.

For FileStorage, damage at pickle level and ZODB level (dangling referenc=
es=20
etc) can be checked with fsrefs.py.=20

For DirectoryStorage, checkds.py checks as much as it can, plus you can r=
un it=20
on a live storage. Im not sure about the situation for BerkelyStorages.

--=20
Toby Dickenson
http://www.geminidataloggers.com/people/tdickenson