[Zope-dev] Coroner's toolkit for zope, or how to figure out what went wrong.

Toby Dickenson tdickenson@geminidataloggers.com
Mon, 12 Aug 2002 17:08:45 +0100


On Monday 12 Aug 2002 4:50 pm, Joachim Werner wrote:
> Hi!
>
> I know of exactly two cases that could really cause a ZODB loose data: =
if
> you reach the 2GB limit with a Python not compiled for larger files and=
 if
> you reach the physical limit of your storage. That is, if your case doe=
sn't
> add a third one ...

FileStorage is robust and mature, but its not a good as this statement=20
suggests.  There have been a number of bugs that cause packing to delete =
more=20
than it should (a few very small holes still remain), bugs that cause=20
FileStorage to overwrite the middle of its log file, and bugs that cause =
its=20
position index to get muddled.

> Have you already tried the usual things, i.e. run fstest.py and/or
> fsrecover.py? It's quite unlikely that you'd loose a whole tree, as the
> data is not physically stored in trees, but added sequentially. You mig=
ht
> have deleted a tree, but that can be rolled back by getting rid of the =
ZODB
> transaction that did the delete.

The first thing I would recommend trying today is shutting down, removing=
=20
data.fs.index, and restarting. In recebnt versions data.fs.index make ver=
y=20
heavy use of BTrees, and all released versions of the BTree code have sma=
ll=20
bugs.


<plug>I am currently developing DirectoryStorage, and one design=20
goal is fault tolerance.        http://dirstorage.sourceforge.net/    </p=
lug>