[ZODB-Dev] ConflictError: database conflict error

[Tim Peters]

[Bob Horvath]
>>> Yea, I saw your email suggesting newer tools might work better and was
>>> in the process of rerunning and debugging using them and literally fell
>>> asleep at the keyboard and never finished it last night.

[Tim Peters]
>> That's no excuse <wink>.
>>
>> I can tell from the output that you're still not using the current
>> (3.2.4) ZODB, so you're not using the best version of fsrefs.py.  Or
>> maybe you're using a current fsrefs.py script, but didn't grab the rest
>> of ZODB 3.2.4?

[Bob]
> Hmm, I thought I had replied to this, but don't see it.  Apologies if I
> am sending twice.
>
> I am trying to use the 3.2.4 ZODB tools. I have a PYTHONPATH including
> the ZODB directory and the python libs from a 2.7.0 version of Zope.

Then something isn't working right.  For example, your fsrefs.out contains
strings like

    oid 00000000000028b4

where under ZODB 3.2.4 they would look like

    oid 0x28b4

instead.  That's a difference in ZODB.utils.oid_repr, which fsrefs.py
imports, not a difference in fsrefs.py iself.  Maybe your PYTHONPATH is
accessing the Zope libraries first -- can't tell from here.  Running Python
with the -v switch will tell you how imports get resolved.


>>> I'll fix up those files later tonight.  An interesting thing is running
>>> the newer tools doesn't yield any "failed to loads", but rather a lot
>>> of "referes to invalid objects".
>>>
>>>
>>> http://files.horvath.com/fsrefs.out  (newer fsrefs)
>>> http://files.horvath.com/fsrefs.old  (older fsrefs)
>>>
>>> If you look at the older one, there are two POSKeyErrors in there that
>>> are not in the newer one.

>> Yes, it's complaining about an object whose creation has been undone --
>> there is no current revision of the object.  Newer versions of fsrefs.py
>> handle that more gracefully.  Note that the two oids in the POSKeyErrors
>> are precisely the same oids as in the two "failed to load" messages.
>> The newer fsrefs don't complain about those, simply because there's
>> nothing wrong about undoing the creation of an object; it was a bug that
>> fsrefs.py used to complain about those.

> OK, so running the new tools yields a lot (57 objects) of "refers to
> invalid objects" (279 objects). 

To be clear here, you also had 57 instances of that under the older tools
(in fsrefs.old's output).  What I hoped for using the newer tools is that
you'd stop getting bogus complaints (like tracebacks for objects whose
creation had been undone); that much seems to have been accomplished.

>  What options are there to fix it?
>
> - edit or fix objects from the /manage interface
> - edit, fix, or deleted from the python command line
> - getting out a hex editor and hacking the file to bits
>
> I've tried the second option but have not been successful at figuring out
> how to make the calls to delete an object.

Someone else has to help you here (anyone?).  As I said before, I've never
had the need to make any repairs like this, so have nothing but "general
principles" to offer -- no actual experience with your flavor of problem.

Because I know a lot about how an .fs file's bits hang together, I'd
probably go the "hack the file to bits" route.  Other people would be more
comfortable with other approaches, but this isn't a common topic on zodb-dev
so I don't know what other people have done.