[ZODB-Dev] Preliminary notes from fixing a bad data.fs

[Tim Peters]

[ethan fremen]
> OK, so I think I've finally gone through and un-wedged this Data.fs.

Congratulations!  It would be OK by me if you advertised here that you're
willing to perform this service for pay <0.5 wink>.

> My first and foremost question is: what is the value of dying cold on a
> pack if an object is unpickleable? 

I don't know why you'd assume this is deliberate behavior.  It seems like
asking what the value is of dying in a pack if you get a MemoryError, or any
other exception that isn't expected.  While I didn't write the pack code, I
strongly doubt it was intended to be used on corrupted files, so that the
author(s) never even considered that unpickling might raise "impossible"
exceptions.

> I modified pack.py

fspack.py, right?  (There is no pack.py that I know of.)

> to just return an empty list if, when searching for references, it was
> unable to pickle

Do you mean pickle there, or unpickle?

> instead of dying- i.e. when it calls referencesf.

Offhand, I don't know whether that's reasonable.  If the effect of your
change is to suppress the exception and keep going, then it sure sounds like
that opens a door to silent data loss.  That would be bad.  But I'm not
really clear enough on what you did to guess.

> Also, perhaps I'm just "special", but fsrecover.py seems to have created
> a Data.fs.index with some bizzare information; at least I got tracebacks
> like:
>
> 2005-01-07T13:12:50 ERROR(200) Zope Couldn't install ExternalMethod
> Traceback (most recent call last):
>    File "/home/emf/zope/lib/python/OFS/Application.py", line 791, in
> install_product
>      get_transaction().commit()
>    File "/home/emf/zope/lib/python/ZODB/Transaction.py", line 241, in
commit
>      ncommitted += self._commit_objects(objects)
>    File "/home/emf/zope/lib/python/ZODB/Transaction.py", line 356, in
> _commit_objects
>      jar.commit(o, self)
>    File "/home/emf/zope/lib/python/ZODB/Connection.py", line 454, in
commit
>      s=dbstore(oid,serial,p,version,transaction)
>    File "/home/emf/zope/lib/python/ZODB/FileStorage.py", line 782, in
store
>      serials=(oserial, serial))
> ConflictError: database conflict error (oid 0x05, serial this txn started
> with 0x00 1900-01-01 00:00:00.000000, serial currently committed
> 0x0339fe733cf5ddf7 2001-01-20 21:39:14.287598)

Woo hoo!  Congratulations again -- for the past couple days we've been
fruitlessly speculating here about how we could possibly get a conflict
error that claimed the "before" serial was 0.  There seem to be a couple
reports of that per year, but doesn't look like anyone ever got anywhere
with it.

If you have a way to reproduce this, opening a bug report would be highly
appreciated.

I haven't rooted around in fsrecover.py's internals, but a quick check now
didn't reveal it doing anything directly to .index files.

> The Zope started fine when I deleted Data.fs.index.

It's peculiar, in part because an .index file doesn't have any info about
serial numbers.  It essentially just maps oids to .fs file offsets.  The
serials are stored in the .fs file.  Maybe, after packing, you got a new .fs
file but were still using the old .index file?  Then the latter would
contain now-bogus offsets.  It would be easy to read up any sort of nonsense
then.  It's going to be hard to tell unless you have a way to reproduce
this.

Speaking of the dangers of silent data loss, here's a bit of the code that
tries to write an .index file:

        try:
            try:
                os.remove(index_name)
            except OSError:
                pass
            os.rename(tmp_name, index_name)
        except: pass

So if anything at all goes wrong in this part (insufficient permission, bugs
in the code that leave index_name or tmp_name wrong, ...), you won't hear
anything about it.
 
> Aside from that, I think I may have wrapped up a few zope-specific
> methods for nuking invalid object references... I'll probably clean them
> up further, and see if it can't be distributed.

Great -- I'm sure people would find that helpful!