[ZODB-Dev] Re: extreme clock skew

Lalo Martins lalo at exoweb.net
Tue Sep 20 20:02:00 EDT 2005


And so says Tim Peters on 21/09/05 07:44...
> [Lalo Martins]
>>A few days later, we realized that pack wasn't working anymore
> 
> What does "wasn't working" mean?

sorry for not giving detail here, I figured it would be obvious.

During the clock skew, the database was packed at least once.

So after time returned to normal, packing refuses to run, claiming "the
database was already packed at a later time".

> Which version of ZODB are you using?  ZODB always intended that tids be
> monotonically increasing, but some older versions have known bugs where that
> can be violated.

That's a Zope 2.7.3; I'm pretty sure it doesn't have these bugs (judging
from the research I did before posting).

> Under any current ZODB release, you should see a
> CRITICAL-level msg logged whenever FileStorage detects a tid more than 30
> minutes in the future.

> So you've ignored critical-level log messages for a few weeks?  I'm not
> trying to pick a fight <wink>, I'm trying to understand how you got into
> this.  A critical-level log msg is the most severe thing we can do without
> plain refusing to run at all.

It does.  And we did.  Don't worry, I already had that fight :-) We have
a procedure in place for reviewing logs, so if it managed to let this
one slip trough the cracks for weeks, it's clearly insufficient.  I'm
going to be reviewing it (the procedure) after I'm out of this mess.

> So-- alas --you can
> probably do dozens of millions transactions more and _still_ be getting
> timestamps in the same one-second interval.

Yeah, I realize that.  :-)

>> [wondering whether the packing fix makes sense]
> You didn't tell us anything about what your packing fix did, so there's no
> way we can guess.

It considers timestamps in the future as infinitely old; so in a
hypotetical database where all transactions are in the future, it would
copy only "reachable" transactions.

>>Soooo... is there any way to recover this data?
> 
> "Recover" seems an odd choice of word here.  Is your data lost or corrupted?
> I didn't get the impression above that it was.  I'll continue to assume that
> it isn't lost or corrupt.

No, the data is fine.  You're right, it's too strong a word.  The only
thing I want to recover is the ability to timestamp objects meaningfully.

> Look at the code for BaseStorage.copyTransactionsFrom().  You should be able
> to fiddle that easily to reset tids to anything you like, while making a
> copy of your Data.fs.  The old Data.fs won't be changed.  The new Data.fs
> will have the tids you force it to have.

Yeah, that's what I was planning to do, thanks for the confirmation.

> In fact, while I haven't tried this, I _suspect_ that changing
>             self.tpc_begin(transaction, None, transaction.status)
>                                         ^^^^

this, however, I didn't find by myself; thanks a lot :-)

best,
                                               Lalo Martins
--
      So many of our dreams at first seem impossible,
       then they seem improbable, and then, when we
       summon the will, they soon become inevitable.
--
http://www.exoweb.net/                  mailto:lalo at exoweb.net
GNU: never give up freedom                 http://www.gnu.org/



More information about the ZODB-Dev mailing list