Who's experiencing FileStorage corruption?
[follow-ups to zodb-dev@zope.org, please] We (Zope Corp) occasionally get reports of FileStorage (Data.fs) corruption of kinds that aren't understood, and that are never seen in Zope Corp's own Zope deployments, neither in extreme artificial stress tests. We take corruption very seriously, and there are no known bugs in the current releases of ZODB that can cause corruption. Therefore we're very keen to investigate cases that may still exist. By "corruption", I mean what corruption conventionally means for any file: the Data.fs and/or Data.fs.index file is damaged at the byte level, as if someone had overwritten some region (or regions) with nonsense bytes. Of course this can be a disaster when it occurs. Visible symptoms may include: + FileStorage.py raises CorruptedDataError. + FileStorage.py passes on this exception from Python's struct module: error: unpack str size does not match format The only currently known causes for these are hardware problems (bad disk, bad disk controller, loose connection, flaky memory chip) and equivalently fatal system software bugs (buggy system disk driver, buggy system I/O libraries, buggy third-party software). If you experience corruption not due to such causes beyond ZODB's control(*), we want to hear about it! The best place to report a case is on the Zope Collector: http://collector.zope.org/Zope with topic "Database". Because database files can be very large, and may contain sensitive data, please don't attach them to the report; you should be willing to let us get copies of them privately, though. (*) How do you know whether the cause is beyond ZODB's control? You probably can't, and that's fine. A case where you can: the last time Jim Fulton tried to track one of these down, the customer's system was in such bad shape that they were unable to make a readable tar file containing their database files: attempts to do so created corrupt *tar* files. If I/O on your system is plain broken, then no, FileStorage isn't going to work either.
participants (1)
-
Tim Peters