Florent Guillaume's database Conflict Error on session objects patch not working ?
Hi all, As discussed here : http://mail.zope.org/pipermail/zope/2007-January/170151.html and continued here : http://mail.zope.org/pipermail/zope/2007-January/170156.html I am experiencing issues regarding a database ConflictError on session objects. I use zope 2.9.0 python 2.4.2, and the code of the MultiDB support has been patched (as suggested by Florent Guillaume) http://mail.zope.org/pipermail/zodb-dev/2006-January/009737.html My main concern is to debug my application where, on every couple of requests coming from two different browsers at the same time (before the first request is , one of them wins and the other always fails and raises a conflict error. It is not a ReadConflictError, nor a WriteConflictError. It is a "ConflictError: database conflict error". I am aware that conflicts occure up to 20% of the time, but in my case it is almost 100% in this specific situation ! If I do not use the session object in my code, I have no errors at all and everything just goes fine. Any idea on how to debug this ? Florent, maybe ? someone help me ! I am stuck on this for 2 weeks now. NOTE : I do not use Zeo, nor frames. Thanks, Y.Chaouche
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 yacine chaouche wrote:
Hi all,
As discussed here : http://mail.zope.org/pipermail/zope/2007-January/170151.html and continued here : http://mail.zope.org/pipermail/zope/2007-January/170156.html
I am experiencing issues regarding a database ConflictError on session objects.
I use zope 2.9.0 python 2.4.2, and the code of the MultiDB support has been patched (as suggested by Florent Guillaume) http://mail.zope.org/pipermail/zodb-dev/2006-January/009737.html
My main concern is to debug my application where, on every couple of requests coming from two different browsers at the same time (before the first request is , one of them wins and the other always fails and raises a conflict error. It is not a ReadConflictError, nor a WriteConflictError. It is a "ConflictError: database conflict error". I am aware that conflicts occure up to 20% of the time, but in my case it is almost 100% in this specific situation !
If I do not use the session object in my code, I have no errors at all and everything just goes fine.
Any idea on how to debug this ? Florent, maybe ? someone help me ! I am stuck on this for 2 weeks now.
NOTE : I do not use Zeo, nor frames.
You might try 'faster', an alternative ZODB-based sessioning implementation: one of its goals is to reduce the chance of collisions compared to the stock implementation: http://agendaless.com/Members/tseaver/software/faster/ Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFw0aj+gerLs4ltQ4RAuDLAJ9VXpLtmnr5EwYMV1AkS1Gxa0lvmwCgoGIv hkn26PUMSmmBKO8nDYFdmkU= =n+Qz -----END PGP SIGNATURE-----
I forward your message to the list. I installed faster : 1) It has no INSTALL file 2) I created a better session manager object under the id newSession, and hit the test button. I Got this error : Traceback (most recent call last): File "/opt/aef/Zope-2.9.0/lib/python/ZPublisher/Publish.py", line 113, in publish request, bind=1) File "/opt/aef/Zope-2.9.0//lib/python/ZPublisher/mapply.py", line 88, in mapply if debug is not None: return debug(object,args,context) File "/opt/aef/Zope-2.9.0/lib/python/ZPublisher/Publish.py", line 40, in call_object result=apply(object,args) # Type s<cr> to step into published object. File "/opt/aef/Zope-2.9.0//lib/python/Shared/DC/Scripts/Bindings.py", line 311, in __call__ return self._bindAndExec(args, kw, None) File "/opt/aef/Zope-2.9.0//lib/python/Shared/DC/Scripts/Bindings.py", line 329, in _bindAndExec def _bindAndExec(self, args, kw, caller_namespace): File "/opt/aef/Zope-2.9.0//lib/python/Shared/DC/Scripts/Bindings.py", line 348, in _bindAndExec return self._exec(bound_data, args, kw) File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/PageTemplateFile.py", line 110, in _exec return self.pt_render(extra_context=bound_names) File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/PageTemplate.py", line 104, in pt_render tal=not source, strictinsert=0)() File "/opt/aef/Zope-2.9.0//lib/python/TAL/TALInterpreter.py", line 234, in __call__ def __call__(self): File "/opt/aef/Zope-2.9.0//lib/python/TAL/TALInterpreter.py", line 238, in __call__ self.interpret(self.program) File "/opt/aef/Zope-2.9.0//lib/python/TAL/TALInterpreter.py", line 281, in interpret handlers[opcode](self, args) File "/opt/aef/Zope-2.9.0//lib/python/TAL/TALInterpreter.py", line 507, in do_setLocal_tal self.engine.setLocal(name, self.engine.evaluateValue(expr)) File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/TALES.py", line 221, in evaluate return expression(self) File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/Expressions.py", line 182, in __call__ def __call__(self, econtext): File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/Expressions.py", line 185, in __call__ return self._eval(econtext) File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/Expressions.py", line 180, in _eval return render(ob, econtext.vars) File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/Expressions.py", line 88, in render raise File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/Expressions.py", line 85, in render ob = ob() File "/opt/aef/Zope-2.9.0/lib/python/Products/faster/sessiondata.py", line 255, in addItemsToSession before = len(session) AttributeError: __len__ Do you use a bug reporting system or something for your product ? Thank you, Y.Chaouche 2007/2/2, Tres Seaver <tseaver@palladion.com>:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
yacine chaouche wrote:
Hi all,
As discussed here : http://mail.zope.org/pipermail/zope/2007-January/170151.html and continued here : http://mail.zope.org/pipermail/zope/2007-January/170156.html
I am experiencing issues regarding a database ConflictError on session objects.
I use zope 2.9.0 python 2.4.2, and the code of the MultiDB support has been patched (as suggested by Florent Guillaume) http://mail.zope.org/pipermail/zodb-dev/2006-January/009737.html
My main concern is to debug my application where, on every couple of requests coming from two different browsers at the same time (before the first request is , one of them wins and the other always fails and raises a conflict error. It is not a ReadConflictError, nor a WriteConflictError. It is a "ConflictError: database conflict error". I am aware that conflicts occure up to 20% of the time, but in my case it is almost 100% in this specific situation !
If I do not use the session object in my code, I have no errors at all and everything just goes fine.
Any idea on how to debug this ? Florent, maybe ? someone help me ! I am stuck on this for 2 weeks now.
NOTE : I do not use Zeo, nor frames.
You might try 'faster', an alternative ZODB-based sessioning implementation: one of its goals is to reduce the chance of collisions compared to the stock implementation:
http://agendaless.com/Members/tseaver/software/faster/
Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFw0aj+gerLs4ltQ4RAuDLAJ9VXpLtmnr5EwYMV1AkS1Gxa0lvmwCgoGIv hkn26PUMSmmBKO8nDYFdmkU= =n+Qz -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 yacine chaouche wrote:
I forward your message to the list.
I installed faster : 1) It has no INSTALL file
You're correct. It installs like other Zope2 products, but it could use some description of how to replace the stock session manager with the new one.
2) I created a better session manager object under the id newSession, and hit the test button. I Got this error :
Assuming that you mean the "Add" button on the "Test" tab, I can't reproduce this in my Zopd 2.9-based sandbox (running on the head of the 2.9 branch). The items get added to the list (assuming that the key is separated from the value on the same line with a space.
Traceback (most recent call last): File "/opt/aef/Zope-2.9.0/lib/python/ZPublisher/Publish.py", line 113, in publish request, bind=1) File "/opt/aef/Zope-2.9.0//lib/python/ZPublisher/mapply.py", line 88, in mapply if debug is not None: return debug(object,args,context) File "/opt/aef/Zope-2.9.0/lib/python/ZPublisher/Publish.py", line 40, in call_object result=apply(object,args) # Type s<cr> to step into published object. File "/opt/aef/Zope-2.9.0//lib/python/Shared/DC/Scripts/Bindings.py", line 311, in __call__ return self._bindAndExec(args, kw, None) File "/opt/aef/Zope-2.9.0//lib/python/Shared/DC/Scripts/Bindings.py", line 329, in _bindAndExec def _bindAndExec(self, args, kw, caller_namespace): File "/opt/aef/Zope-2.9.0//lib/python/Shared/DC/Scripts/Bindings.py", line 348, in _bindAndExec return self._exec(bound_data, args, kw) File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/PageTemplateFile.py", line 110, in _exec return self.pt_render(extra_context=bound_names) File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/PageTemplate.py", line 104, in pt_render tal=not source, strictinsert=0)() File "/opt/aef/Zope-2.9.0//lib/python/TAL/TALInterpreter.py", line 234, in __call__ def __call__(self): File "/opt/aef/Zope-2.9.0//lib/python/TAL/TALInterpreter.py", line 238, in __call__ self.interpret(self.program) File "/opt/aef/Zope-2.9.0//lib/python/TAL/TALInterpreter.py", line 281, in interpret handlers[opcode](self, args) File "/opt/aef/Zope-2.9.0//lib/python/TAL/TALInterpreter.py", line 507, in do_setLocal_tal self.engine.setLocal(name, self.engine.evaluateValue(expr)) File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/TALES.py", line 221, in evaluate return expression(self) File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/Expressions.py", line 182, in __call__ def __call__(self, econtext): File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/Expressions.py", line 185, in __call__ return self._eval(econtext) File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/Expressions.py", line 180, in _eval return render(ob, econtext.vars) File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/Expressions.py", line 88, in render raise File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/Expressions.py", line 85, in render ob = ob() File "/opt/aef/Zope-2.9.0/lib/python/Products/faster/sessiondata.py", line 255, in addItemsToSession before = len(session) AttributeError: __len__
Do you use a bug reporting system or something for your product ?
Yes, there is a collector linked of the product's homepage. http://agendaless.com/Members/tseaver/software/faster/ I would note that we have had the product deployed in production for two customers for six months now, without any issues at all. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFw2qz+gerLs4ltQ4RAn4nAJ9I5kHhVtcdHHjLKQhiPpZH0/YyJwCdFrV1 X5aHzRbZHQVi1lGLH/Hkpsw= =ufay -----END PGP SIGNATURE-----
I added a new faster session object in my root folder. Once it was created, i was on the property manager form and clicked on the test tab and got the error reported above. Is the request["SESSION"] object that you use in your code addressing the classic SESSION object (in temp_folder/session_data_container) or does it address your new session data objects ? for the latter case, maybe i should rename the classic (zope's) SESSION object to SESSION2 or something... Y.Chaouche 2007/2/2, Tres Seaver <tseaver@palladion.com>:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
yacine chaouche wrote:
I forward your message to the list.
I installed faster : 1) It has no INSTALL file
You're correct. It installs like other Zope2 products, but it could use some description of how to replace the stock session manager with the new one.
2) I created a better session manager object under the id newSession, and hit the test button. I Got this error :
Assuming that you mean the "Add" button on the "Test" tab, I can't reproduce this in my Zopd 2.9-based sandbox (running on the head of the 2.9 branch). The items get added to the list (assuming that the key is separated from the value on the same line with a space.
Traceback (most recent call last): File "/opt/aef/Zope-2.9.0/lib/python/ZPublisher/Publish.py", line 113, in publish request, bind=1) File "/opt/aef/Zope-2.9.0//lib/python/ZPublisher/mapply.py", line 88, in mapply if debug is not None: return debug(object,args,context) File "/opt/aef/Zope-2.9.0/lib/python/ZPublisher/Publish.py", line 40, in call_object result=apply(object,args) # Type s<cr> to step into published object. File "/opt/aef/Zope-2.9.0//lib/python/Shared/DC/Scripts/Bindings.py", line 311, in __call__ return self._bindAndExec(args, kw, None) File "/opt/aef/Zope-2.9.0//lib/python/Shared/DC/Scripts/Bindings.py", line 329, in _bindAndExec def _bindAndExec(self, args, kw, caller_namespace): File "/opt/aef/Zope-2.9.0//lib/python/Shared/DC/Scripts/Bindings.py", line 348, in _bindAndExec return self._exec(bound_data, args, kw) File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/PageTemplateFile.py", line 110, in _exec return self.pt_render(extra_context=bound_names) File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/PageTemplate.py", line 104, in pt_render tal=not source, strictinsert=0)() File "/opt/aef/Zope-2.9.0//lib/python/TAL/TALInterpreter.py", line 234, in __call__ def __call__(self): File "/opt/aef/Zope-2.9.0//lib/python/TAL/TALInterpreter.py", line 238, in __call__ self.interpret(self.program) File "/opt/aef/Zope-2.9.0//lib/python/TAL/TALInterpreter.py", line 281, in interpret handlers[opcode](self, args) File "/opt/aef/Zope-2.9.0//lib/python/TAL/TALInterpreter.py", line 507, in do_setLocal_tal self.engine.setLocal(name, self.engine.evaluateValue(expr)) File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/TALES.py", line 221, in evaluate return expression(self) File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/Expressions.py", line 182, in __call__ def __call__(self, econtext): File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/Expressions.py", line 185, in __call__ return self._eval(econtext) File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/Expressions.py", line 180, in _eval return render(ob, econtext.vars) File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/Expressions.py", line 88, in render raise File "/opt/aef/Zope-2.9.0//lib/python/Products/PageTemplates/Expressions.py", line 85, in render ob = ob() File "/opt/aef/Zope-2.9.0/lib/python/Products/faster/sessiondata.py", line 255, in addItemsToSession before = len(session) AttributeError: __len__
Do you use a bug reporting system or something for your product ?
Yes, there is a collector linked of the product's homepage.
http://agendaless.com/Members/tseaver/software/faster/
I would note that we have had the product deployed in production for two customers for six months now, without any issues at all.
Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFFw2qz+gerLs4ltQ4RAn4nAJ9I5kHhVtcdHHjLKQhiPpZH0/YyJwCdFrV1 X5aHzRbZHQVi1lGLH/Hkpsw= =ufay -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2 Feb 2007, at 17:45, Tres Seaver wrote:
I installed faster : 1) It has no INSTALL file
You're correct. It installs like other Zope2 products, but it could use some description of how to replace the stock session manager with the new one.
There is a INSTALL.txt on the CVS HEAD, which I put there a while ago. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iD8DBQFFw3GQRAx5nvEhZLIRAlKaAJ9coVr4xAlj2UF2h5Rd3dAcF7RYpwCfbo9r gvK73yvVkZgOUFU/4QV1ehU= =BpQY -----END PGP SIGNATURE-----
yacine chaouche wrote at 2007-2-2 14:39 +0100:
... As discussed here : http://mail.zope.org/pipermail/zope/2007-January/170151.html and continued here : http://mail.zope.org/pipermail/zope/2007-January/170156.html
I am experiencing issues regarding a database ConflictError on session objects.
Hm, I have issues with your problem descriptions... The referenced links above do not refer to conflict errors but to "XMLRPC and strace". Then, too, I had great difficulties to understand your problem...
... It is not a ReadConflictError, nor a WriteConflictError. It is a "ConflictError: database conflict error".
A "ConflictError" is in fact a "WriteConflictError".
I am aware that conflicts occure up to 20% of the time, but in my case it is almost 100% in this specific situation !
You can easily achieve almost sure conflicts. We did once with a frame based solution: All frames in a frameset accessed the session and tried to store some information in it. As the browser requested the frames concurrently, there was a very high probablity for conflicts. In our case, there was no real need for most of the frames to access the session. Removing the unnecessary accesses removed the problem. But, if the frames had a real need to store something in the session, we would have to use persistent subobjects in the session to separate the access of the various frames into different objects.
If I do not use the session object in my code, I have no errors at all and everything just goes fine.
Then, you are using some add-on component that *is* using the session object. If the session is not used, there will be no session related conflict error.
Any idea on how to debug this ? Florent, maybe ? someone help me ! I am stuck on this for 2 weeks now.
You could try to delete "/temp_folder/session_data" temporarily (it will be recreated on the next Zope startup) to find out which component does use sessions. -- Dieter
Hi Dieter, What part of what I said don't you understand ? I will try to make it clearer here :
The referenced links above do not refer to conflict errors but to "XMLRPC and strace".
Then, too, I had great difficulties to understand your problem...
Yes, when I posted the XMLRPC and strace message on the list I did not know yet the publishing process and everything. So I thought that zope just went crazy pretending to receive new requests. In reality, it was the same request that was retried over and over again until no exception was raised. Once I understood the publishing process, I edited the publish.py file to display a traceback of every raised exception before retrying the request. Thus, I could see that there were database conflict errors that were raised.
I am aware that conflicts occure up to 20% of the time, but in my case it is almost 100% in this specific situation !
You can easily achieve almost sure conflicts.
We did once with a frame based solution:
I do NOT use frames.
If I do not use the session object in my code, I have no errors at all and everything just goes fine.
Then, you are using some add-on component that *is* using the session object.
That's what I am saying : in my code I need to use the session to store objects. When I use the session, in my code, not in a plugin or so, in my own code, I have errors. When I do not use session, in my own code, I have no errors. This is supposed to prove that the conflict error comes from the use of the session object in my code. This is not supposed to prove that another plugin is using it anyway... Thanks, Y.Chaouche 2007/2/2, Dieter Maurer <dieter@handshake.de>:
yacine chaouche wrote at 2007-2-2 14:39 +0100:
... As discussed here : http://mail.zope.org/pipermail/zope/2007-January/170151.html and continued here : http://mail.zope.org/pipermail/zope/2007-January/170156.html
I am experiencing issues regarding a database ConflictError on session objects.
Hm, I have issues with your problem descriptions...
The referenced links above do not refer to conflict errors but to "XMLRPC and strace".
Then, too, I had great difficulties to understand your problem...
... It is not a ReadConflictError, nor a WriteConflictError. It is a "ConflictError: database conflict error".
A "ConflictError" is in fact a "WriteConflictError".
I am aware that conflicts occure up to 20% of the time, but in my case it is almost 100% in this specific situation !
You can easily achieve almost sure conflicts.
We did once with a frame based solution:
All frames in a frameset accessed the session and tried to store some information in it.
As the browser requested the frames concurrently, there was a very high probablity for conflicts.
In our case, there was no real need for most of the frames to access the session. Removing the unnecessary accesses removed the problem.
But, if the frames had a real need to store something in the session, we would have to use persistent subobjects in the session to separate the access of the various frames into different objects.
If I do not use the session object in my code, I have no errors at all and everything just goes fine.
Then, you are using some add-on component that *is* using the session object.
If the session is not used, there will be no session related conflict error.
Any idea on how to debug this ? Florent, maybe ? someone help me ! I am stuck on this for 2 weeks now.
You could try to delete "/temp_folder/session_data" temporarily (it will be recreated on the next Zope startup) to find out which component does use sessions.
-- Dieter
yacine chaouche wrote at 2007-2-3 13:55 +0100:
... Once I understood the publishing process, I edited the publish.py file to display a traceback of every raised exception before retrying the request. Thus, I could see that there were database conflict errors that were raised.
Usually, such conflicts are logged -- without a need to change "Publish.py".
... When I do not use session, in my own code, I have no errors. This is supposed to prove that the conflict error comes from the use of the session object in my code. This is not supposed to prove that another plugin is using it anyway...
Sorry, I did not read your message carefully enough. -- Dieter
On 2/3/07, yacine chaouche <yacinechaouche@gmail.com> wrote:
I am experiencing issues regarding a database ConflictError on session objects.
Can you detail all the various getSesion and setSession methods of: http://mail.zope.org/pipermail/zope/2007-January/170167.html
I use zope 2.9.0 python 2.4.2.
It's worth trying the latest of each of those. Cheers Michael
participants (5)
-
Dieter Maurer -
Jens Vagelpohl -
Michael Dunstan -
Tres Seaver -
yacine chaouche