[ZODB-Dev] ConflictError: database conflict error

[Bob Horvath]

 > [Bob Horvath]
 > ...
 >> fsrefs is not silent.  Two objects "failed to load" and there were 279
 >> "missing".
 >>
 >> ...
 >>
 >> The output from them are at...
 >>
 >> http://files.horvath.com/fstest.out
 >> http://files.horvath.com/fsrefs.out
 >
 > There's no "failed to load" output in that file.  If you run fsrefs with
 > -v,
 > it will show a traceback when something fails to load, and that gives 
more
 > clues.  You were running an old-enough version of fsrefs that it would
 > give
 > spurious "failed to load" messages for objects whose creation had merely
 > been undone, though (that's a bug, repaired in ZODB 3.2.4).
 >
 >> http://files.horvath.com/fsdump.out  (very large)
 >
 > Sorry, that file doesn't exist.

Yea, I saw your email suggesting newer tools might work better and was 
in the process of rerunning and debugging using them and literally fell 
asleep at the keyboard and never finished it last night.


 >
 >> So what do you do with  a file like that?
 >
 > Staring at it until your eyes bleed is a good start <0.8 wink>.  All your
 > "missing objects" are BTrees, Buckets, and TreeSets.  It's impossible for
 > me
 > to guess what they're being used for, especially since fsdump.out doesn't
 > exist.

I'll fix up those files later tonight.  An interesting thing is running 
the newer tools doesn't yield any "failed to loads", but rather a lot of 
"referes to invalid objects".


http://files.horvath.com/fsrefs.out  (newer fsrefs)
http://files.horvath.com/fsrefs.old  (older fsrefs)

If you look at the older one, there are two POSKeyErrors in there that 
are not in the newer one. 

 >
 > In generally, you can search fsdump.py's output for the oids in
 > fsrefs.py's
 > output, and hope to guess what purpose these objects served.  With luck,
 > they were created and modified by transactions that added a comment, and
 > fsdump shows those comments (when they exist).  For example, it's
 > *possible*
 > (I'm not claiming it's likely) that all these missing objects are simply
 > parts of catalogs or indices that could be recreated by rebuilding them
 > from
 > scratch. 

Hmmm, interesting. I'll work on cleaning things up.


 > Don't know, and can't tell from here.  That's where your
 > knowledge
 > of your app comes in.

Thanks for the glimmer of hope, and sorry for wasting your time on 
posting links and the screwing them up.


Using the older tools, comparing the fsrefs output above, with the older 
fsdump (which is up there at http://files.horvath.com/fsdump.old), the 
two things that threw PosKeyError are 0xcbc2 and 0xcbc3 did have some 
undo interactions.