[Zope-DB] Who's experiencing FileStorage corruption?

[Tim Peters]

[follow-ups to zodb-dev at zope.org, please]

We (Zope Corp) occasionally get reports of FileStorage (Data.fs) corruption
of kinds that aren't understood, and that are never seen in Zope Corp's own
Zope deployments, neither in extreme artificial stress tests.  We take
corruption very seriously, and there are no known bugs in the current
releases of ZODB that can cause corruption.  Therefore we're very keen to
investigate cases that may still exist.

By "corruption", I mean what corruption conventionally means for any file:
the Data.fs and/or Data.fs.index file is damaged at the byte level, as if
someone had overwritten some region (or regions) with nonsense bytes.  Of
course this can be a disaster when it occurs.  Visible symptoms may include:

+ FileStorage.py raises CorruptedDataError.

+ FileStorage.py passes on this exception from Python's struct module:

    error: unpack str size does not match format

The only currently known causes for these are hardware problems (bad disk,
bad disk controller, loose connection, flaky memory chip) and equivalently
fatal system software bugs (buggy system disk driver, buggy system I/O
libraries, buggy third-party software).

If you experience corruption not due to such causes beyond ZODB's
control(*), we want to hear about it!  The best place to report a case is on
the Zope Collector:

    http://collector.zope.org/Zope

with topic "Database".  Because database files can be very large, and may
contain sensitive data, please don't attach them to the report; you should
be willing to let us get copies of them privately, though.


(*)
How do you know whether the cause is beyond ZODB's control?  You probably
can't, and that's fine.  A case where you can:  the last time Jim Fulton
tried to track one of these down, the customer's system was in such bad
shape that they were unable to make a readable tar file containing their
database files:  attempts to do so created corrupt *tar* files.  If I/O on
your system is plain broken, then no, FileStorage isn't going to work
either.