[Warning] Potential serious Session/TemporaryStorage problem
Recently, strange session related "Conflict Errors" have been reported. The strange thing about these conflicts is that the timestamp when the object was read from the ZODB is *AFTER* the timestamp the object has at commit time. Apparently, the object state jumped backward in time. When the object is modified, this results in a "ConflictError". However, when the object is only read, then old data is used -- probably without anyone noticing this. This might be a reason for the many inconsistencies Zope's Session implementation is plagued with. Here is the respective collector entry: http://collector.zope.org/Zope/1488 I will try to localize the place where this jumping backward in time happens. -- Dieter
Dieter Maurer wrote at 2004-9-1 20:45 +0200:
... Recently, strange session related "Conflict Errors" have been reported.
The strange thing about these conflicts is that the timestamp when the object was read from the ZODB is *AFTER* the timestamp the object has at commit time. Apparently, the object state jumped backward in time. When the object is modified, this results in a "ConflictError". However, when the object is only read, then old data is used -- probably without anyone noticing this. ... I will try to localize the place where this jumping backward in time happens.
The implementors of "ZODB.FileStorage.FileStorage.store" and "tempstorage.TemporaryStorage.TemporaryStorage.store" have simply been a bit confused about what is old and what is new: the two values are interchanged: "was" is the serial at commit time, "now" is the serial at load time. Thus, the "was serial" must be larger than the "now serial". -- Dieter
Dieter -- Which version of Filestorage/TemporaryStorage has this error? Are you going to do a patch? I'm also tracking a problem with Temporary Stroage which might be related. I use session variables rather heavily and, occasionally, a running session loses its variables. The problem manifests itself as a KeyError when referencing the session. Other sessionss in the same Zope appear unaffected. And it is not a timeout problem. On Wed, 1 Sep 2004, Dieter Maurer wrote:
Dieter Maurer wrote at 2004-9-1 20:45 +0200:
... Recently, strange session related "Conflict Errors" have been reported.
The strange thing about these conflicts is that the timestamp when the object was read from the ZODB is *AFTER* the timestamp the object has at commit time. Apparently, the object state jumped backward in time. When the object is modified, this results in a "ConflictError". However, when the object is only read, then old data is used -- probably without anyone noticing this. ... I will try to localize the place where this jumping backward in time happens.
The implementors of "ZODB.FileStorage.FileStorage.store" and "tempstorage.TemporaryStorage.TemporaryStorage.store" have simply been a bit confused about what is old and what is new: the two values are interchanged: "was" is the serial at commit time, "now" is the serial at load time. Thus, the "was serial" must be larger than the "now serial".
-- Dieter _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Dennis Allison wrote at 2004-9-1 14:44 -0700:
Dieter --
Which version of Filestorage/TemporaryStorage has this error? Are you going to do a patch?
As I have reported in the message you answered to, this is only a display problem: I have not discovered a serious problem -- the problem appears only in the "ConflictError" message. The two serials have been interchanged.
I'm also tracking a problem with Temporary Stroage which might be related. I use session variables rather heavily and, occasionally, a running session loses its variables. The problem manifests itself as a KeyError when referencing the session. Other sessionss in the same Zope appear unaffected. And it is not a timeout problem.
Until today, I thought this were impossible. However, today, a colleague reported that he has seen occurrencies where individual session variables have disappeared (while others did not). He is has analysed the access logs and is sure that all session variable have been set. Thus, almost surely, something is wrong with sessions or temporary storage. We just do not yet know what. -- Dieter
I think I have tracked down the problem here. It was one program that did not follow the usual copy-out modify copy-in mechanism required to trigger persistent stores (see Transcience API in The Zope Book). I fixed that and the problem seems to have (mostly) gone away although we are still testing. On Thu, 2 Sep 2004, Dieter Maurer wrote:
Dennis Allison wrote at 2004-9-1 14:44 -0700:
Dieter --
Which version of Filestorage/TemporaryStorage has this error? Are you going to do a patch?
As I have reported in the message you answered to, this is only a display problem: I have not discovered a serious problem -- the problem appears only in the "ConflictError" message. The two serials have been interchanged.
I'm also tracking a problem with Temporary Stroage which might be related. I use session variables rather heavily and, occasionally, a running session loses its variables. The problem manifests itself as a KeyError when referencing the session. Other sessionss in the same Zope appear unaffected. And it is not a timeout problem.
Until today, I thought this were impossible. However, today, a colleague reported that he has seen occurrencies where individual session variables have disappeared (while others did not). He is has analysed the access logs and is sure that all session variable have been set.
Thus, almost surely, something is wrong with sessions or temporary storage. We just do not yet know what.
-- Dieter _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Dennis Allison wrote:
I think I have tracked down the problem here. It was one program that did not follow the usual copy-out modify copy-in mechanism required to trigger persistent stores (see Transcience API in The Zope Book).
Where is this in the Zope Book? I haven't had any session problems but I'd like to catch this early in cae I do in the future... Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
In the API Appendix under Transcience Interface and also in the product development documents where the persistence mechanism is discussed. On Fri, 3 Sep 2004, Chris Withers wrote:
Dennis Allison wrote:
I think I have tracked down the problem here. It was one program that did not follow the usual copy-out modify copy-in mechanism required to trigger persistent stores (see Transcience API in The Zope Book).
Where is this in the Zope Book? I haven't had any session problems but I'd like to catch this early in cae I do in the future...
Chris
-- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
participants (3)
-
Chris Withers -
Dennis Allison -
Dieter Maurer