[ZODB-Dev] RelStorage pack with history-free storage results in POSKeyErrors

[Chris Withers]

On 27/01/2011 11:24, Shane Hathaway wrote:
> On 01/27/2011 03:57 AM, Chris Withers wrote:
>> It would also be *really* handy if zodbpack could run off a normal
>> zope.conf for both logging and storage config (although, in my case, I'd
>> then need to be able to specify which storage to pack, I want to avoid
>> packing one of them!)
>
> Please note that the zodbpack utility is simple, short, and has no extra
> dependencies.  I prefer to keep it that way.  If you want to do
> something more interesting, please fork zodbpack.

Fair enough, simple is good for me :-)
If errors or logging occur inside zodbpack, where do they go?

>> However, more than happy to poke, just tell me where and for what...
>
> Is the problem repeatable?

Yes.

> If you start with the same database twice
> and pack, do you end up with POSKeyErrors on the same OIDs?

Not sure.

> I know
> that's probably a long test to run, but I'm not yet sure what else to
> suggest.

Okay, some more info...

I re-converted the .fs file for this storage.
This time, I've taken a mysqldump backup.

I then packed, and *then* started the batch process: no problems.
While the batch was running, I tried packing again but the pack finished 
without any problems.

Okay, so next up I restored the backup.
This time I set the batch running and *then* started the pack.
This eventually caused the POSKeyErrors to begin again.

I'm going to have a go at knocking up a small batch script that you can 
run to reproduce the issue, but there's definitely an issue here when 
packing while changing lots of data in a ZODB.

cheers,

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk