[Zope-dev] [Bug] FileStorage packing looses data

[Dieter Maurer]

ATT: cross post

We have a nasty FileStorage packing bug

   <http://collector.zope.org/Zope/875>

The bug is caused by a strange handling of backpointers
during pack's copying phase: the code checks whether the
backpointer goes to a packed or unpacked position.
When it goes to a packed record, it guesses what new value
the backpointer should have -- and it guesses wrong.
This can lead to data loss!


Currently, "pack" treats its two phase quite asymmetrically
with respect to position resolution:

  Packing phase:

    backpointers are resolved
    "prev" and "nvprev" positions are determined via "index/nvindex"

  Copying phase:
    backpointers are not resolved but adjusted depending on whether
      they point to packed or unpacked records
    "prev" and "nvprev" are only adjusted via "index/nvindex"
    when they point to packed data. Otherwise they are
    adjusted via a constant offset.


I do not see any reason why this should not be symmetrical:

  all backpointers resolved
  all positions determined via "index/nvindex"

  No more guessing, no longer an offset that needs to be held constant.

  What is correct for packed data should be correct for not
  yet packed data, as well.


Do you see an error in this argument?


Dieter