[ZODB-Dev] FileIterator: Unused/wrong edge case

[Christian Theune]

Hi,

we're working on the last item on our to-do list regarding storage 
iteration: having iterators pick up transactions that were written after 
the iteration started.

In the `FileIterator` class there exists the following code:

----
if pos + h.tlen + 8 > self._file_size:
     # Hm, the data were truncated or the checkpoint flag wasn't
     # cleared.  They may also be corrupted,
     # in which case, we don't want to totally lose the data.
     logger.warning("%s truncated, possibly due to"
                    " damaged records at %s", self._file.name, pos)
     break
----

We've pondered this for a bit and think that the described case about 
truncation doesn't make sense: If the file really was truncated in an 
invalid way, this will (or at least should) be detected by the 
FileStorageFormatter code.

Additionally, we don't see the point in keeping a reference to the size 
of the file as it was at the beginning of the iteration. This seems an 
arbitrary limit whose intention is neither documented nor required by 
any interface.

We'd like to remove the check displayed above.

Christian and Thomas

-- 
gocept gmbh & co. kg - forsterstrasse 29 - 06112 halle (saale) - germany
www.gocept.com - ct at gocept.com - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development