Hi, Just by chance, I pressed the reload button of my browser for quite some time, so that my Zope3-based site was reloaded over and over. I was surprised to find the following in the logs: 2008-10-20T18:45:15 WARNING ZopePublication Competing writes/reads at /BSPSite/act/s/info: database conflict error (oid 0x0733, class zope.session.session.SessionData) Can someone explain me why this happens, if this is something dangerous or if I can forget about it? I tested the site with Apache Bench (ab2), with many concurrent requests and could not reproduce the problem. So my wild guess is the following: I use session.PersistentSessionDataContainer for my session, so every request will probably read/write to some Zope3 container/btree. And then, for some reasons these reads/writes compete, which leads to the above warning. If I use AB2, there's no cookie, thus no session, hence the problem does not arise. On the other hand, the zope root site (the Rotterdam ZMI) seems not to have this problem, as it seems, I can't trigger this warning message with fast reloads. I'm unsure if I should just ignore that or do something about it. Any hints? Best Regards, Hermann -- hermann@qwer.tk GPG key ID: 299893C7 (on keyservers) FP: 0124 2584 8809 EF2A DBF9 4902 64B4 D16B 2998 93C7
Hermann Himmelbauer wrote:
Hi, Just by chance, I pressed the reload button of my browser for quite some time, so that my Zope3-based site was reloaded over and over. I was surprised to find the following in the logs:
2008-10-20T18:45:15 WARNING ZopePublication Competing writes/reads at /BSPSite/act/s/info: database conflict error (oid 0x0733, class zope.session.session.SessionData)
Can someone explain me why this happens, if this is something dangerous or if I can forget about it?
I tested the site with Apache Bench (ab2), with many concurrent requests and could not reproduce the problem.
So my wild guess is the following: I use session.PersistentSessionDataContainer for my session, so every request will probably read/write to some Zope3 container/btree. And then, for some reasons these reads/writes compete, which leads to the above warning. If I use AB2, there's no cookie, thus no session, hence the problem does not arise.
On the other hand, the zope root site (the Rotterdam ZMI) seems not to have this problem, as it seems, I can't trigger this warning message with fast reloads.
Presumably that site does not use sessioning so does not suffer from the same problem. Laurence
On Mon, Oct 20, 2008 at 06:56:39PM +0200, Hermann Himmelbauer wrote:
Just by chance, I pressed the reload button of my browser for quite some time, so that my Zope3-based site was reloaded over and over. I was surprised to find the following in the logs:
2008-10-20T18:45:15 WARNING ZopePublication Competing writes/reads at /BSPSite/act/s/info: database conflict error (oid 0x0733, class zope.session.session.SessionData)
Can someone explain me why this happens, if this is something dangerous or if I can forget about it?
This happens whenever two parallel transactions try to modify the same persistent object at the same time. The first one of those will succeed, the other will raise ConflictError, and then the Zope publisher will emit this warning to the log and retry the request, up to a certain number of times (either 3 or 5, I don't remember exactly). It is mostly harmless. If you get many conflict errors, performance will suffer, and if you get too many, eventually the 3 (or 5) retry limit will not be sufficient and your end-users will see system error pages.
I tested the site with Apache Bench (ab2), with many concurrent requests and could not reproduce the problem.
Interesting. Could you reproduce the problem by pressing the reload button again? Sometimes Zope uses a pattern of creating a nonexistent object on first access. This can cause a conflict error if two requests try to create that object, but it will happen only once.
I use session.PersistentSessionDataContainer for my session, so every request will probably read/write to some Zope3 container/btree. And then, for some reasons these reads/writes compete, which leads to the above warning. If I use AB2, there's no cookie, thus no session, hence the problem does not arise.
On the other hand, the zope root site (the Rotterdam ZMI) seems not to have this problem, as it seems, I can't trigger this warning message with fast reloads.
I'm unsure if I should just ignore that or do something about it.
It's a warning. It would be better if it did not occur, but sometimes that's hard to achieve and the efforts do not pay off. HTH, Marius Gedminas -- Hoping the problem magically goes away by ignoring it is the "microsoft approach to programming" and should never be allowed. -- Linus Torvalds
Am Mittwoch 22 Oktober 2008 00:03:51 schrieb Marius Gedminas:
On Mon, Oct 20, 2008 at 06:56:39PM +0200, Hermann Himmelbauer wrote:
Just by chance, I pressed the reload button of my browser for quite some time, so that my Zope3-based site was reloaded over and over. I was surprised to find the following in the logs:
2008-10-20T18:45:15 WARNING ZopePublication Competing writes/reads at /BSPSite/act/s/info: database conflict error (oid 0x0733, class zope.session.session.SessionData)
Can someone explain me why this happens, if this is something dangerous or if I can forget about it?
This happens whenever two parallel transactions try to modify the same persistent object at the same time. The first one of those will succeed, the other will raise ConflictError, and then the Zope publisher will emit this warning to the log and retry the request, up to a certain number of times (either 3 or 5, I don't remember exactly).
It is mostly harmless. If you get many conflict errors, performance will suffer, and if you get too many, eventually the 3 (or 5) retry limit will not be sufficient and your end-users will see system error pages.
Ah, I see, now I understand it.
I tested the site with Apache Bench (ab2), with many concurrent requests and could not reproduce the problem.
Interesting. Could you reproduce the problem by pressing the reload button again?
Yes, I can reproduce this any time. But I looked into it further, and it's quite weird: - If I access the site without having logged in (= a new cookie/session is created), it does not happen. - If I login, pressing F5 rapidly triggers these warnings. - BUT: If I logout and press F5, the warnings are still there. - If I close the browser/reopen it (-> New session), the warning is gone. So it seems, once the session stores/has stored login information, this object leads to a conflict. Interesting - but I still don't know what this object is. And as Apache bench does not use sessions (as it has no cookies), the problem never arises.
Sometimes Zope uses a pattern of creating a nonexistent object on first access. This can cause a conflict error if two requests try to create that object, but it will happen only once.
Hmmm, I don't really understand what this means - how does that happen, and why would Zope create a nonexistent object? Anyway, many thanks for your help, I'm quite relieved that this is no serious problem! Best Regards, Hermann -- hermann@qwer.tk GPG key ID: 299893C7 (on keyservers) FP: 0124 2584 8809 EF2A DBF9 4902 64B4 D16B 2998 93C7
Hermann Himmelbauer schrieb:
Yes, I can reproduce this any time. But I looked into it further, and it's quite weird:
- If I access the site without having logged in (= a new cookie/session is created), it does not happen. - If I login, pressing F5 rapidly triggers these warnings. - BUT: If I logout and press F5, the warnings are still there. - If I close the browser/reopen it (-> New session), the warning is gone.
So it seems, once the session stores/has stored login information, this object leads to a conflict. Interesting - but I still don't know what this object is.
You should inspect what happens in zope.session to find out what happens there. From your description and the purpose of sessions probably this happens: Everytime you access you site the session is checked for authentication informations. Sessions have a set timeout. zope.session has to write a new timestamp into the session so it can drop the session after the timeout. This can cause conflicts. When you log out the authentication informations are removed from the session, but the cookie an the server side session are still there. The timestamp is updated with every request. If you close your browser the cookie is removed by the browser. Until you authenticate zope does not create a new session/set a new cookie. I'm quite sure that you can't get conflicts with zope.session if you use two different sessions at the same time and conflicts with one session are rare enough and with zope's conflict resolution no problem (unless you press F5 really fast) ..Carsten
What version of zope.session are you using? A fix (using zope.minmax) for ConflictErrors related to the update of access time on sessionData object was committed into zope.session in rev76899. Not sure if that solves this problem of conflict errors that you see, but if you are using a version of zope.session prior to r76899, upgrading may make those problems go away. Regards, Satchit On Oct 22, 2008, at 10:27 AM, Carsten Senger wrote:
Hermann Himmelbauer schrieb:
Yes, I can reproduce this any time. But I looked into it further, and it's quite weird:
- If I access the site without having logged in (= a new cookie/ session is created), it does not happen. - If I login, pressing F5 rapidly triggers these warnings. - BUT: If I logout and press F5, the warnings are still there. - If I close the browser/reopen it (-> New session), the warning is gone.
So it seems, once the session stores/has stored login information, this object leads to a conflict. Interesting - but I still don't know what this object is.
You should inspect what happens in zope.session to find out what happens there. From your description and the purpose of sessions probably this happens:
Everytime you access you site the session is checked for authentication informations. Sessions have a set timeout. zope.session has to write a new timestamp into the session so it can drop the session after the timeout. This can cause conflicts. When you log out the authentication informations are removed from the session, but the cookie an the server side session are still there. The timestamp is updated with every request. If you close your browser the cookie is removed by the browser. Until you authenticate zope does not create a new session/set a new cookie.
I'm quite sure that you can't get conflicts with zope.session if you use two different sessions at the same time and conflicts with one session are rare enough and with zope's conflict resolution no problem (unless you press F5 really fast)
..Carsten
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Am Mittwoch 22 Oktober 2008 16:44:58 schrieb Satchidanand Haridas:
What version of zope.session are you using?
A fix (using zope.minmax) for ConflictErrors related to the update of access time on sessionData object was committed into zope.session in rev76899. Not sure if that solves this problem of conflict errors that you see, but if you are using a version of zope.session prior to r76899, upgrading may make those problems go away.
Hmmm, I'm using zope.session from the KGS, so I'm unsure which revision I have. However, when looking at the "$Id:" lines in the python files, I can see e.g. the following line: tests.py:$Id: tests.py 80055 2007-09-25 21:28:54Z rogerineichen That probably indicates, that my revision > r76899, is that true? Best Regards, Hermann -- hermann@qwer.tk GPG key ID: 299893C7 (on keyservers) FP: 0124 2584 8809 EF2A DBF9 4902 64B4 D16B 2998 93C7
On Oct 23, 2008, at 5:27 AM, Hermann Himmelbauer wrote:
Am Mittwoch 22 Oktober 2008 16:44:58 schrieb Satchidanand Haridas:
What version of zope.session are you using?
A fix (using zope.minmax) for ConflictErrors related to the update of access time on sessionData object was committed into zope.session in rev76899. Not sure if that solves this problem of conflict errors that you see, but if you are using a version of zope.session prior to r76899, upgrading may make those problems go away.
Hmmm, I'm using zope.session from the KGS, so I'm unsure which revision I have. However, when looking at the "$Id:" lines in the python files, I can see e.g. the following line:
tests.py:$Id: tests.py 80055 2007-09-25 21:28:54Z rogerineichen
That probably indicates, that my revision > r76899, is that true?
Yes you have higher revision. However, there was another issue that was fixed in -r 87351. The following line should be there: $Id: tests.py 87351 2008-06-12 21:02:43Z gary $ That version was released immediately in -r 87352 as zope.session-3.5.2 and the change log states: version 3.5.2 (2008-06-12) -------------------------- - Remove ConflictErrors caused on SessionData caused by setting ``lastAccessTime``. Using zope.session-3.5.2 is worth the try. Zvezdan
Am Donnerstag 23 Oktober 2008 15:31:05 schrieb Zvezdan Petkovic:
On Oct 23, 2008, at 5:27 AM, Hermann Himmelbauer wrote:
Am Mittwoch 22 Oktober 2008 16:44:58 schrieb Satchidanand Haridas:
What version of zope.session are you using?
A fix (using zope.minmax) for ConflictErrors related to the update of access time on sessionData object was committed into zope.session in rev76899. Not sure if that solves this problem of conflict errors that you see, but if you are using a version of zope.session prior to r76899, upgrading may make those problems go away. version 3.5.2 (2008-06-12)
--------------------------
- Remove ConflictErrors caused on SessionData caused by setting ``lastAccessTime``.
Using zope.session-3.5.2 is worth the try.
Yep, that was worth the try! As far as I can tell from now, the warning message is gone now. Many thanks! Best Regards, Hermann -- hermann@qwer.tk GPG key ID: 299893C7 (on keyservers) FP: 0124 2584 8809 EF2A DBF9 4902 64B4 D16B 2998 93C7
participants (6)
-
Carsten Senger -
Hermann Himmelbauer -
Laurence Rowe -
Marius Gedminas -
Satchidanand Haridas -
Zvezdan Petkovic