[ZODB-Dev] FileIterator: Unused/wrong edge case

[Christian Theune]

Note:

The specification of storage iteration is gonna change so it explicitly 
states that it's going to represent the set of transactions as seen from 
the point where iteration started.

Christian Theune schrieb:
> Hi,
> 
> we're working on the last item on our to-do list regarding storage 
> iteration: having iterators pick up transactions that were written after 
> the iteration started.
> 
> In the `FileIterator` class there exists the following code:
> 
> ----
> if pos + h.tlen + 8 > self._file_size:
>     # Hm, the data were truncated or the checkpoint flag wasn't
>     # cleared.  They may also be corrupted,
>     # in which case, we don't want to totally lose the data.
>     logger.warning("%s truncated, possibly due to"
>                    " damaged records at %s", self._file.name, pos)
>     break
> ----
> 
> We've pondered this for a bit and think that the described case about 
> truncation doesn't make sense: If the file really was truncated in an 
> invalid way, this will (or at least should) be detected by the 
> FileStorageFormatter code.

We still wonder whether the code is necessary and this case wouldn't be 
picked up by some of the FileStorageFormatter code.

> Additionally, we don't see the point in keeping a reference to the size 
> of the file as it was at the beginning of the iteration. This seems an 
> arbitrary limit whose intention is neither documented nor required by 
> any interface.

The limitation is no there.

> We'd like to remove the check displayed above.

We don't anymore.

Christian and Thomas

-- 
gocept gmbh & co. kg - forsterstrasse 29 - 06112 halle (saale) - germany
www.gocept.com - ct at gocept.com - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development