[ZODB-Dev] RelStorage pack with history-free storage results in POSKeyErrors

[Shane Hathaway]

On 01/26/2011 11:52 AM, Chris Withers wrote:
> On 26/01/2011 14:08, Shane Hathaway wrote:
>> I've checked in a fix in Subversion.  Please try it out.  I need to look
>> at the possible pack issue recently reported before we make a release.
>
> Where is this pack issue documented/discussed?

See the discussion here with Anton Stonor.  We are still only 
hypothesizing that there's a bug.

> Also, does RelStorage have a bug tracker anywhere?

Not yet.  The need for one has not been clear until very recently. 
RelStorage is turning into a community project and every community 
project needs a bug tracker.  I suggest we use Launchpad.

> I have a main ZODB storage in a normal RelStorage with several
> BTreeFolder2's mounted into it that come from a history free storage.
> I'm using Zope 2.12.7 with ZODB 3.9.6
>
> I attempted to pack the history-free storage with the following config:
>
> <relstorage destination>
>      pack-gc true
>
>      <mysql>
>      ...
>      </mysql>
>      keep-history false
> </relstorage>
>
> The pack was going on while a batch process was pouring data into the
> ZODB by way for a loop with one transaction per iteration.
>
> The pack succeeded fine with no errors but about 10 minutes after the
> pack started, the batch process started throwing POSKeyErrors of the
> following sort:
>
>    File
> "/var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZODB/Connection.py", line
> 897, in _setstate
>       p, serial = self._storage.load(obj._p_oid, '')
>     File
> "/var/buildout-eggs/RelStorage-1.4.0-py2.6.egg/relstorage/storage.py",
> line 462, in load
>       raise POSKeyError(oid)
> POSKeyError: 0xf30dbd
>
> ------ 2011-01-26T10:26:00 WARNING
> POSKeyError on oid 15925081: no tid found; history-free adapter
>
> ------ 2011-01-26T10:26:00 ERROR
> Couldn't load state for 0xf2ff59
> Traceback (most recent call last):
>     File
> "/var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZODB/Connection.py", line
> 847, in setstate
>       self._setstate(obj)
>     File
> "/var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZODB/Connection.py", line
> 897, in _setstate
>       p, serial = self._storage.load(obj._p_oid, '')
>     File
> "/var/buildout-eggs/RelStorage-1.4.0-py2.6.egg/relstorage/storage.py",
> line 462, in load
>       raise POSKeyError(oid)
> POSKeyError: 0xf2ff59
>
> It looks like these errors are occurring through the web client to this
> ZODB as well now.
>
> Any ideas what's going on here? What more useful information can I provide?

That may be related, but first, are you mounting databases?  You have to 
be careful with mounted databases and packing.

Shane