[ZODB-Dev] Eek!

[Jim Fulton]

Chris Withers wrote:
> 
> > Jeremy Hylton wrote:
> > >
> > >   CW> Would I be particularly out of order in stating that this
> > >   CW> appears to be a shocking oversight in FileStorage?
> >
> > You wouldn't be out of order, but I wouldn't agree with you in the
> abstract.
> 
> Huh? Sorry, lost me after the comma there...

I don't think it would be a shocking oversight if FileStorage
wasted some disk space for aborted transactions if they were
cleaned up with a pack.

> > This specifica case is a pretty bad bug.
> 
> This is what I thought ;-) It's mighty worrying to suddenly find you hit a
> 2GB limit on a Data.fs which should only be a coupla hundred MB but has been
> experiencing a lot of failed transactions...

Actually, it's not nearly as bad as I thought.

First, this only affects the case where an error 
occurs after a successful tpc_vote and before
tpc_finish. If an error occurs during a vote, 
which is the only time the file should become too large, 
the file is truncated correctly.

Further, the bug should not cause the transaction to be left.
If Zope were restarted before any other transactions were written, 
it would truncate the bad data on startup.

Otherwise, subsequent transactions will overwrite the aborted
transactions, because the file position hasn't been advanced
past the bad transaction.

> > > I assume that a pack removes this data.  If it doesn't, then that's a
> > > shocking oversight.
> >
> > It does, but it throws away any subsequent transactions.
> 
> OK, now this is _really_ worrying me, under what specific circumstances
> would you see subsequent transactions being lost? How can I fix this
> problem?

Actually, this situation can't occur. In rereading the code in preparation
for fixing the bug, I realized that my initial analysis of the situation was
incorrect. Sorry for the false alarm. 

Jim

--
Jim Fulton           mailto:jim@zope.com       Python Powered!        
CTO                  (888) 344-4332            http://www.python.org  
Zope Corporation     http://www.zope.com       http://www.zope.org