Zope 2.8 and conflict errors
Hi! Does anybody tried 2.8 and can say something about conflict errors? I've done few tests and results are not good. We've checked one of our applications on Zope 2.7.2 and the same application on Zope 2.8. We have simulated 50-65 users executing the same path in application. There was total count 1500 pages visited during test. With Zope 2.7.2 we've received about 39 internal server errors HTTP/1.1 500. Errors in Zope 2.7.2 are mainly ReadConflictErrors with DCOracle2.DA.Procedure like: ------ 2005-08-01T09:54:08 INFO(0) TM:65540 Failed to abort object Traceback (most recent call last): File "/opt/Zope/2.7.2/lib/python/ZODB/Transaction.py", line 393, in _commit_error j.abort(o, self) File "/opt/Zope/2.7.2/lib/python/ZODB/Connection.py", line 561, in setstate invalid = self._is_invalidated(obj) File "/opt/Zope/2.7.2/lib/python/ZODB/Connection.py", line 590, in _is_invalidated raise ReadConflictError(object=obj) ReadConflictError: database read conflict error (oid 00000000000299de, class Products.DCOracle2.DA.Procedure) ------ Other errors are like "Shouldn't load state xxx when connection is closed" but these are known bugs in older ZODB version. Zope 2.8 supports MVCC so we hoped that the number of errors decrease but... with Zope 2.8 we get about 129 internal server errors HTTP/1.1 500 and additionally a lot of 402 errors (unauthorized). Errors 402 are because of internal server errors on login page (testing application continues test without checking if user has been properly logged in). In the event.log there are no conflict errors saved :-/ , only unauthorized. We get the error from client responses and from console. I don't know what happens. May it be because of using old data.fs file (from 2.7.2) with Zope 2.8? Anybody has similiar experiences? Is Zope 2.8 useable?? Below few errors from the console (runzope) and from user browser: ---------------------------------------------------------------- 2005-08-01 10:46:22 INFO ZODB conflict error at /crProd/acl_users/login (16 conflicts since startup at 2005-08-01T10:45:58) 2005-08-01 10:46:22 BLATHER Conflict traceback Traceback (most recent call last): File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in publish request, bind=1) File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply if debug is not None: return debug(object,args,context) File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in call_object result=apply(object,args) # Type s<cr> to step into published object. File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 311, in __call__ return self._bindAndExec(args, kw, None) File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 348, in _bindAndExec return self._exec(bound_data, args, kw) File "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", line 323, in _exec result = f(*args, **kw) File "Script (Python)", line 3, in login File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1214, in __getattr__ v = self.get(key, default, returnTaints=returnTaints) File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1174, in get if callable(v): v = v() File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 93, in getSessionData return self._getSessionDataObject(key) File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 180, in _getSessionDataObject ob = container.new_or_existing(key) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 844, in new_or_existing item = self.get(key, _marker) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 491, in get item = self._move_item(k, current_ts, default) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 297, in _move_item abucket = self._data.get(ts, None) # XXX ReadConflictError hotspot File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in setstate self._setstate(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in _setstate self._load_before_or_conflict(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in _load_before_or_conflict raise ReadConflictError(object=obj) ReadConflictError: database read conflict error (oid 0x0a, class BTrees._IOBTree.IOBTree) ---------------------------------------------------------------- error in /crProd/acl_users/login - is with request.SESSION statement. ---------------------------------------------------------------- 2005-08-01 10:46:21 BLATHER Conflict traceback Traceback (most recent call last): File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in publish request, bind=1) File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply if debug is not None: return debug(object,args,context) File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in call_object result=apply(object,args) # Type s<cr> to step into published object. File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 311, in __call__ return self._bindAndExec(args, kw, None) File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 348, in _bindAndExec return self._exec(bound_data, args, kw) File "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", line 323, in _exec result = f(*args, **kw) File "Script (Python)", line 3, in login File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1214, in __getattr__ v = self.get(key, default, returnTaints=returnTaints) File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1174, in get if callable(v): v = v() File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 93, in getSessionData return self._getSessionDataObject(key) File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 180, in _getSessionDataObject ob = container.new_or_existing(key) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 844, in new_or_existing item = self.get(key, _marker) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 491, in get item = self._move_item(k, current_ts, default) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 280, in _move_item self._housekeep(current_ts) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 939, in _housekeep self._finalize(now) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 547, in _finalize last_finalized = self._last_finalized_timeslice() File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 1076, in __call__ return self.value File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in setstate self._setstate(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in _setstate self._load_before_or_conflict(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in _load_before_or_conflict raise ReadConflictError(object=obj) ReadConflictError: database read conflict error (oid 0x09, class Products.Transience.Transience.Increaser) ----------------------------------------------------------- or ----------------------------------------------------------- 2005-08-01 10:46:21 BLATHER Conflict traceback Traceback (most recent call last): File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in publish request, bind=1) File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply if debug is not None: return debug(object,args,context) File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in call_object result=apply(object,args) # Type s<cr> to step into published object. File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 311, in __call__ return self._bindAndExec(args, kw, None) File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 348, in _bindAndExec return self._exec(bound_data, args, kw) File "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", line 323, in _exec result = f(*args, **kw) File "Script (Python)", line 3, in login File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1214, in __getattr__ v = self.get(key, default, returnTaints=returnTaints) File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1174, in get if callable(v): v = v() File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 93, in getSessionData return self._getSessionDataObject(key) File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 180, in _getSessionDataObject ob = container.new_or_existing(key) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 847, in new_or_existing self[key] = item File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 435, in __setitem__ length = self._length() # XXX ReadConflictError hotspot File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 1119, in __call__ return self.value File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in setstate self._setstate(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in _setstate self._load_before_or_conflict(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in _load_before_or_conflict raise ReadConflictError(object=obj) ReadConflictError: database read conflict error (oid 0x08, class Products.Transience.Transience.Length2) ----------------------------------------------------------- or ----------------------------------------------------------- 2005-08-01 10:46:21 BLATHER Conflict traceback Traceback (most recent call last): File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in publish request, bind=1) File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply if debug is not None: return debug(object,args,context) File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in call_object result=apply(object,args) # Type s<cr> to step into published object. File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 311, in __call__ return self._bindAndExec(args, kw, None) File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 348, in _bindAndExec return self._exec(bound_data, args, kw) File "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", line 323, in _exec result = f(*args, **kw) File "Script (Python)", line 3, in login File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1214, in __getattr__ v = self.get(key, default, returnTaints=returnTaints) File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1174, in get if callable(v): v = v() File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 93, in getSessionData return self._getSessionDataObject(key) File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 180, in _getSessionDataObject ob = container.new_or_existing(key) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 844, in new_or_existing item = self.get(key, _marker) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 491, in get item = self._move_item(k, current_ts, default) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 297, in _move_item abucket = self._data.get(ts, None) # XXX ReadConflictError hotspot File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in setstate self._setstate(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in _setstate self._load_before_or_conflict(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in _load_before_or_conflict raise ReadConflictError(object=obj) ReadConflictError: database read conflict error (oid 0x0a, class BTrees._IOBTree.IOBTree) ----------------------------------------------------------- or ----------------------------------------------------------- 2005-08-01 10:47:55 INFO ZODB conflict error at /crProd/cr/centralne/procName (87 conflicts since startup at 2005-08-01T10:45:58) 2005-08-01 10:47:55 BLATHER Conflict traceback Traceback (most recent call last): File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 119, in publish transactions_manager.commit() File "/opt/Zope/2.8.0/lib/python/Zope2/App/startup.py", line 215, in commit transaction.commit() File "/opt/Zope/2.8.0/lib/python/transaction/_manager.py", line 84, in commit self.get().commit(sub) File "/opt/Zope/2.8.0/lib/python/transaction/_transaction.py", line 382, in commit self._saveCommitishError() # This raises! File "/opt/Zope/2.8.0/lib/python/transaction/_transaction.py", line 380, in commit self._commitResources() File "/opt/Zope/2.8.0/lib/python/transaction/_transaction.py", line 428, in _commitResources rm.tpc_vote(self) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 637, in tpc_vote s = vote(transaction) File "/opt/Zope/2.8.0/lib/python/ZEO/ClientStorage.py", line 893, in tpc_vote return self._check_serials() File "/opt/Zope/2.8.0/lib/python/ZEO/ClientStorage.py", line 877, in _check_serials raise s ConflictError: database conflict error (oid 0x0299de, class Products.DCOracle2.DA.Procedure, serial this txn started with 0x035f0726c7bfca88 2005-08-01 08:38:46.816215, serial currently committed 0x035f072fed78ebee 2005-08-01 08:47:55.657582) ----------------------------------------------------------- User receives errors like: ----------------------------------------------------------------- Response: HTTP/1.1 500 Internal Server Error Server: Zope/(Zope 2.8.0-final, python 2.3.5, linux2) ZServer/1.1 Date: Fri, 29 Jul 2005 12:53:00 GMT Bobo-Exception-Line: 877 Content-Length: 2020 Bobo-Exception-Value: See the server error log for details Content-Language: pl Bobo-Exception-File: ClientStorage.py Bobo-Exception-Type: ZODB.POSException.ConflictError (...) <TD WIDTH="90%"> <H2>Site Error</H2> <P>An error was encountered while publishing this resource. </P> <P><STRONG>ZODB.POSException.ConflictError</STRONG></P> Sorry, a site error occurred.<p><p>Traceback (innermost last): <ul> Module ZPublisher.Publish, line 187, in publish_module_standard <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 151, in publish</li> <li> Module Zope2.App.startup, line 158, in zpublisher_exception_hook</li> <li> Module ZPublisher.Publish, line 119, in publish</li> <li> Module Zope2.App.startup, line 215, in commit</li> <li> Module transaction._manager, line 84, in commit</li> <li> Module transaction._transaction, line 382, in commit</li> <li> Module transaction._transaction, line 380, in commit</li> <li> Module transaction._transaction, line 428, in _commitResources</li> <li> Module ZODB.Connection, line 637, in tpc_vote</li> <li> Module ZEO.ClientStorage, line 893, in tpc_vote</li> <li> Module ZEO.ClientStorage, line 877, in _check_serials</li> </ul>ConflictError: database conflict error (oid 0x0299de, class Products.DCOracle2.DA.Procedure, serial this txn started with 0x035ef744f3d9b366 2005-07-29 12:52:57.152436, serial currently committed 0x035ef744fe4886cc 2005-07-29 12:52:59.597650) </p> <HR NOSHADE> <P>Troubleshooting Suggestions</P> (...) ----------------------------------------------------------------- -- Maciej Wisniowski
Hi Maciej, I can't speak to the 500 errors you're receiving but Zope 2.8.0 ships with a "TemporaryStorage" implementation (used by sessions in the default configuration) which does not implement MVCC. The one in 2.8.1 (now in beta) and better will implement MVCC, so you *should* see performance improve (or at least see fewer conflict errors while using sessioning). I'm keenly interested in your results, so please let me know. - C On Mon, 2005-08-01 at 12:21 +0200, Maciej Wisniowski wrote:
Hi!
Does anybody tried 2.8 and can say something about conflict errors?
I've done few tests and results are not good.
We've checked one of our applications on Zope 2.7.2 and the same application on Zope 2.8. We have simulated 50-65 users executing the same path in application. There was total count 1500 pages visited during test.
With Zope 2.7.2 we've received about 39 internal server errors HTTP/1.1 500.
Errors in Zope 2.7.2 are mainly ReadConflictErrors with DCOracle2.DA.Procedure like:
------ 2005-08-01T09:54:08 INFO(0) TM:65540 Failed to abort object Traceback (most recent call last): File "/opt/Zope/2.7.2/lib/python/ZODB/Transaction.py", line 393, in _commit_error j.abort(o, self) File "/opt/Zope/2.7.2/lib/python/ZODB/Connection.py", line 561, in setstate invalid = self._is_invalidated(obj) File "/opt/Zope/2.7.2/lib/python/ZODB/Connection.py", line 590, in _is_invalidated raise ReadConflictError(object=obj) ReadConflictError: database read conflict error (oid 00000000000299de, class Products.DCOracle2.DA.Procedure) ------
Other errors are like "Shouldn't load state xxx when connection is closed" but these are known bugs in older ZODB version.
Zope 2.8 supports MVCC so we hoped that the number of errors decrease but... with Zope 2.8 we get about 129 internal server errors HTTP/1.1 500 and additionally a lot of 402 errors (unauthorized).
Errors 402 are because of internal server errors on login page (testing application continues test without checking if user has been properly logged in).
In the event.log there are no conflict errors saved :-/ , only unauthorized. We get the error from client responses and from console.
I don't know what happens. May it be because of using old data.fs file (from 2.7.2) with Zope 2.8?
Anybody has similiar experiences? Is Zope 2.8 useable??
Below few errors from the console (runzope) and from user browser:
---------------------------------------------------------------- 2005-08-01 10:46:22 INFO ZODB conflict error at /crProd/acl_users/login (16 conflicts since startup at 2005-08-01T10:45:58) 2005-08-01 10:46:22 BLATHER Conflict traceback Traceback (most recent call last): File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in publish request, bind=1) File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply if debug is not None: return debug(object,args,context) File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in call_object result=apply(object,args) # Type s<cr> to step into published object. File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 311, in __call__ return self._bindAndExec(args, kw, None) File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 348, in _bindAndExec return self._exec(bound_data, args, kw) File "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", line 323, in _exec result = f(*args, **kw) File "Script (Python)", line 3, in login File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1214, in __getattr__ v = self.get(key, default, returnTaints=returnTaints) File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1174, in get if callable(v): v = v() File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 93, in getSessionData return self._getSessionDataObject(key) File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 180, in _getSessionDataObject ob = container.new_or_existing(key) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 844, in new_or_existing item = self.get(key, _marker) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 491, in get item = self._move_item(k, current_ts, default) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 297, in _move_item abucket = self._data.get(ts, None) # XXX ReadConflictError hotspot File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in setstate self._setstate(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in _setstate self._load_before_or_conflict(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in _load_before_or_conflict raise ReadConflictError(object=obj) ReadConflictError: database read conflict error (oid 0x0a, class BTrees._IOBTree.IOBTree) ----------------------------------------------------------------
error in /crProd/acl_users/login - is with request.SESSION statement.
---------------------------------------------------------------- 2005-08-01 10:46:21 BLATHER Conflict traceback Traceback (most recent call last): File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in publish request, bind=1) File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply if debug is not None: return debug(object,args,context) File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in call_object result=apply(object,args) # Type s<cr> to step into published object. File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 311, in __call__ return self._bindAndExec(args, kw, None) File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 348, in _bindAndExec return self._exec(bound_data, args, kw) File "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", line 323, in _exec result = f(*args, **kw) File "Script (Python)", line 3, in login File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1214, in __getattr__ v = self.get(key, default, returnTaints=returnTaints) File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1174, in get if callable(v): v = v() File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 93, in getSessionData return self._getSessionDataObject(key) File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 180, in _getSessionDataObject ob = container.new_or_existing(key) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 844, in new_or_existing item = self.get(key, _marker) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 491, in get item = self._move_item(k, current_ts, default) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 280, in _move_item self._housekeep(current_ts) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 939, in _housekeep self._finalize(now) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 547, in _finalize last_finalized = self._last_finalized_timeslice() File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 1076, in __call__ return self.value File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in setstate self._setstate(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in _setstate self._load_before_or_conflict(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in _load_before_or_conflict raise ReadConflictError(object=obj) ReadConflictError: database read conflict error (oid 0x09, class Products.Transience.Transience.Increaser) -----------------------------------------------------------
or
----------------------------------------------------------- 2005-08-01 10:46:21 BLATHER Conflict traceback Traceback (most recent call last): File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in publish request, bind=1) File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply if debug is not None: return debug(object,args,context) File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in call_object result=apply(object,args) # Type s<cr> to step into published object. File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 311, in __call__ return self._bindAndExec(args, kw, None) File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 348, in _bindAndExec return self._exec(bound_data, args, kw) File "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", line 323, in _exec result = f(*args, **kw) File "Script (Python)", line 3, in login File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1214, in __getattr__ v = self.get(key, default, returnTaints=returnTaints) File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1174, in get if callable(v): v = v() File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 93, in getSessionData return self._getSessionDataObject(key) File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 180, in _getSessionDataObject ob = container.new_or_existing(key) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 847, in new_or_existing self[key] = item File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 435, in __setitem__ length = self._length() # XXX ReadConflictError hotspot File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 1119, in __call__ return self.value File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in setstate self._setstate(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in _setstate self._load_before_or_conflict(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in _load_before_or_conflict raise ReadConflictError(object=obj) ReadConflictError: database read conflict error (oid 0x08, class Products.Transience.Transience.Length2) -----------------------------------------------------------
or
----------------------------------------------------------- 2005-08-01 10:46:21 BLATHER Conflict traceback Traceback (most recent call last): File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in publish request, bind=1) File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply if debug is not None: return debug(object,args,context) File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in call_object result=apply(object,args) # Type s<cr> to step into published object. File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 311, in __call__ return self._bindAndExec(args, kw, None) File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 348, in _bindAndExec return self._exec(bound_data, args, kw) File "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", line 323, in _exec result = f(*args, **kw) File "Script (Python)", line 3, in login File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1214, in __getattr__ v = self.get(key, default, returnTaints=returnTaints) File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1174, in get if callable(v): v = v() File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 93, in getSessionData return self._getSessionDataObject(key) File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 180, in _getSessionDataObject ob = container.new_or_existing(key) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 844, in new_or_existing item = self.get(key, _marker) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 491, in get item = self._move_item(k, current_ts, default) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 297, in _move_item abucket = self._data.get(ts, None) # XXX ReadConflictError hotspot File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in setstate self._setstate(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in _setstate self._load_before_or_conflict(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in _load_before_or_conflict raise ReadConflictError(object=obj) ReadConflictError: database read conflict error (oid 0x0a, class BTrees._IOBTree.IOBTree) -----------------------------------------------------------
or
----------------------------------------------------------- 2005-08-01 10:47:55 INFO ZODB conflict error at /crProd/cr/centralne/procName (87 conflicts since startup at 2005-08-01T10:45:58) 2005-08-01 10:47:55 BLATHER Conflict traceback Traceback (most recent call last): File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 119, in publish transactions_manager.commit() File "/opt/Zope/2.8.0/lib/python/Zope2/App/startup.py", line 215, in commit transaction.commit() File "/opt/Zope/2.8.0/lib/python/transaction/_manager.py", line 84, in commit self.get().commit(sub) File "/opt/Zope/2.8.0/lib/python/transaction/_transaction.py", line 382, in commit self._saveCommitishError() # This raises! File "/opt/Zope/2.8.0/lib/python/transaction/_transaction.py", line 380, in commit self._commitResources() File "/opt/Zope/2.8.0/lib/python/transaction/_transaction.py", line 428, in _commitResources rm.tpc_vote(self) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 637, in tpc_vote s = vote(transaction) File "/opt/Zope/2.8.0/lib/python/ZEO/ClientStorage.py", line 893, in tpc_vote return self._check_serials() File "/opt/Zope/2.8.0/lib/python/ZEO/ClientStorage.py", line 877, in _check_serials raise s ConflictError: database conflict error (oid 0x0299de, class Products.DCOracle2.DA.Procedure, serial this txn started with 0x035f0726c7bfca88 2005-08-01 08:38:46.816215, serial currently committed 0x035f072fed78ebee 2005-08-01 08:47:55.657582) -----------------------------------------------------------
User receives errors like: ----------------------------------------------------------------- Response: HTTP/1.1 500 Internal Server Error Server: Zope/(Zope 2.8.0-final, python 2.3.5, linux2) ZServer/1.1 Date: Fri, 29 Jul 2005 12:53:00 GMT Bobo-Exception-Line: 877 Content-Length: 2020 Bobo-Exception-Value: See the server error log for details Content-Language: pl Bobo-Exception-File: ClientStorage.py Bobo-Exception-Type: ZODB.POSException.ConflictError
(...)
<TD WIDTH="90%"> <H2>Site Error</H2> <P>An error was encountered while publishing this resource. </P> <P><STRONG>ZODB.POSException.ConflictError</STRONG></P> Sorry, a site error occurred.<p><p>Traceback (innermost last): <ul>
Module ZPublisher.Publish, line 187, in publish_module_standard <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 151, in publish</li> <li> Module Zope2.App.startup, line 158, in zpublisher_exception_hook</li> <li> Module ZPublisher.Publish, line 119, in publish</li> <li> Module Zope2.App.startup, line 215, in commit</li> <li> Module transaction._manager, line 84, in commit</li> <li> Module transaction._transaction, line 382, in commit</li> <li> Module transaction._transaction, line 380, in commit</li> <li> Module transaction._transaction, line 428, in _commitResources</li> <li> Module ZODB.Connection, line 637, in tpc_vote</li> <li> Module ZEO.ClientStorage, line 893, in tpc_vote</li> <li> Module ZEO.ClientStorage, line 877, in _check_serials</li> </ul>ConflictError: database conflict error (oid 0x0299de, class Products.DCOracle2.DA.Procedure, serial this txn started with 0x035ef744f3d9b366 2005-07-29 12:52:57.152436, serial currently committed 0x035ef744fe4886cc 2005-07-29 12:52:59.597650) </p> <HR NOSHADE> <P>Troubleshooting Suggestions</P> (...) -----------------------------------------------------------------
I can't speak to the 500 errors you're receiving but Zope 2.8.0 500 is HTTP code, not the error count. I've written this in a messy way.
Zope 2.8.0 ships with a "TemporaryStorage" implementation (used by sessions in the default configuration) which does not implement MVCC. The one in 2.8.1 (now in beta) and better will implement MVCC, so you *should* see performance improve (or at least see fewer conflict errors while using sessioning). Today I wanted to install 2.8.1 but we had no time... :)
Thanks for the info -- Maciej Wisniowski
I'm keenly interested in your results, so please let me know. Hi!
We have just executed two tests of our app on Zope 2.8.1b1 and the results are good now. 1. Application with SQLRelay backend connected to Oracle. Simulation of 50 users. No internal server errors :) 2. Application with DCOracle2 backend connected to Oracle. Simulation of 50 users. No internal server errors :) Seems good, but we need more tests. Previous installation with Zope 2.8 was on ZEO, this one is standalone Zope 2.8.1. I'm not sure if this matters. Both DCOracle2 and SQLRelay had some changes by me. BTW. SQLRelay version is a bit faster than DCOracle version :) -- Maciej Wisniowski
On Tue, Aug 02, 2005 at 11:28:40AM +0200, Maciej Wisniowski wrote:
Previous installation with Zope 2.8 was on ZEO, this one is standalone Zope 2.8.1. I'm not sure if this matters.
It might. ZEO adds network overhead to ZODB transactions, which generally increases the likelihood of conflict errors. -- Paul Winkler http://www.slinkp.com
It might. ZEO adds network overhead to ZODB transactions, which generally increases the likelihood of conflict errors.
I've created ZEO and executed the same test with my application. Tests were done with SQLRelay backend (DCOracle2 is running now ;)) I have tested this with 50, 55, 60 and 70 users. with 50 users: no 'HTTP/1.1 500' errors with 55 users: no 'HTTP/1.1 500' errors with 60 users: no 'HTTP/1.1 500' 1 error for 1800 urls requested with 65 users: no 'HTTP/1.1 500' 4 errors for 1950 urls requested All errors are with OOBTree and as I can see with tempstorage.TemporaryStorage. They look like: <li> Module ZPublisher.Publish, line 187, in publish_module_standard</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 151, in publish</li> <li> Module Zope2.App.startup, line 158, in zpublisher_exception_hook</li> <li> Module ZPublisher.Publish, line 119, in publish</li> <li> Module Zope2.App.startup, line 215, in commit</li> <li> Module transaction._manager, line 84, in commit</li> <li> Module transaction._transaction, line 381, in commit</li> <li> Module transaction._transaction, line 379, in commit</li> <li> Module transaction._transaction, line 424, in _commitResources</li> <li> Module ZODB.Connection, line 462, in commit</li> <li> Module ZODB.Connection, line 503, in _commit</li> <li> Module ZODB.Connection, line 526, in _store_objects</li> <li> Module tempstorage.TemporaryStorage, line 200, in store</li> </ul>ConflictError: database conflict error (oid 0x0944, class BTrees._OOBTree.OOBTree, serial this txn started with 0x035f134dc11a2977 2005-08-03 12:29:45.258327, serial currently committed 0x035f1357051c7199 2005-08-03 12:39:01.197916) So far I see that DCOracle2 version of my app caused 9 errors for the same scenario with 50 users. These are conflict errors with DCOracle2.DA.Procedure and in ZEO.ClientStorage... -- Maciej Wisniowski
I've lost context here... is this good or bad? ;-) On Wed, 2005-08-03 at 16:32 +0200, Maciej Wisniowski wrote:
It might. ZEO adds network overhead to ZODB transactions, which generally increases the likelihood of conflict errors.
I've created ZEO and executed the same test with my application.
Tests were done with SQLRelay backend (DCOracle2 is running now ;))
I have tested this with 50, 55, 60 and 70 users. with 50 users: no 'HTTP/1.1 500' errors with 55 users: no 'HTTP/1.1 500' errors with 60 users: no 'HTTP/1.1 500' 1 error for 1800 urls requested with 65 users: no 'HTTP/1.1 500' 4 errors for 1950 urls requested
All errors are with OOBTree and as I can see with tempstorage.TemporaryStorage. They look like:
<li> Module ZPublisher.Publish, line 187, in publish_module_standard</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 151, in publish</li> <li> Module Zope2.App.startup, line 158, in zpublisher_exception_hook</li> <li> Module ZPublisher.Publish, line 119, in publish</li> <li> Module Zope2.App.startup, line 215, in commit</li> <li> Module transaction._manager, line 84, in commit</li> <li> Module transaction._transaction, line 381, in commit</li> <li> Module transaction._transaction, line 379, in commit</li> <li> Module transaction._transaction, line 424, in _commitResources</li> <li> Module ZODB.Connection, line 462, in commit</li> <li> Module ZODB.Connection, line 503, in _commit</li> <li> Module ZODB.Connection, line 526, in _store_objects</li> <li> Module tempstorage.TemporaryStorage, line 200, in store</li> </ul>ConflictError: database conflict error (oid 0x0944, class BTrees._OOBTree.OOBTree, serial this txn started with 0x035f134dc11a2977 2005-08-03 12:29:45.258327, serial currently committed 0x035f1357051c7199 2005-08-03 12:39:01.197916)
So far I see that DCOracle2 version of my app caused 9 errors for the same scenario with 50 users. These are conflict errors with DCOracle2.DA.Procedure and in ZEO.ClientStorage...
I've lost context here... is this good or bad? ;-)
I'm keenly interested in your results, so please let me know.
I've just send my results ;) Good or bad... it depends. It's much better than it was with Zope 2.8.0, it is better than with Zope 2.7.2 but it is not perfect because errors are still here. Our application will have about 500 users or more. Of course they won't be working simultaneously and app will be running on much faster machines than one used for tests but I suppose that some errors will appear. For me these results seems really good. Another good thing is that the SQLRelayDA is going to be more stable than DCOracle2 and is even faster than DCO2. I have almost ready products, scripts and changes to SQLRelayDA that can convert application from DCO2 to SQLRelay with one click and we are going to resign from DCO2, but still more testing is necessary. Previously you've written that: "Zope 2.8.0 ships with a "TemporaryStorage" implementation (used by sessions in the default configuration) which does not implement MVCC. The one in 2.8.1 (now in beta) and better will implement MVCC" so I hope that maybe next Zope 2.8.x release will be using MVCC for OOBTree (if this is the point?). Does anybody know something about that? -- Maciej Wisniowski
On Wed, 2005-08-03 at 23:02 +0200, Maciej Wisniowski wrote:
Previously you've written that:
"Zope 2.8.0 ships with a "TemporaryStorage" implementation (used by sessions in the default configuration) which does not implement MVCC. The one in 2.8.1 (now in beta) and better will implement MVCC"
so I hope that maybe next Zope 2.8.x release will be using MVCC for OOBTree (if this is the point?). Does anybody know something about that?
MVCC only really relates to storages. Individual database objects don't participate in MVCC. If you put your OOBTrees in a storage that supports MVCC, you get it "for free". - C
On 2005-08-04, at 02:32 GMT+12:00, Maciej Wisniowski wrote:
All errors are with OOBTree and as I can see with tempstorage.TemporaryStorage. They look like:
<li> Module ZPublisher.Publish, line 187, in publish_module_standard</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 151, in publish</li> <li> Module Zope2.App.startup, line 158, in zpublisher_exception_hook</li> <li> Module ZPublisher.Publish, line 119, in publish</li> <li> Module Zope2.App.startup, line 215, in commit</li> <li> Module transaction._manager, line 84, in commit</li> <li> Module transaction._transaction, line 381, in commit</li> <li> Module transaction._transaction, line 379, in commit</li> <li> Module transaction._transaction, line 424, in _commitResources</li> <li> Module ZODB.Connection, line 462, in commit</li> <li> Module ZODB.Connection, line 503, in _commit</li> <li> Module ZODB.Connection, line 526, in _store_objects</li> <li> Module tempstorage.TemporaryStorage, line 200, in store</li> </ul>ConflictError: database conflict error (oid 0x0944, class BTrees._OOBTree.OOBTree, serial this txn started with 0x035f134dc11a2977 2005-08-03 12:29:45.258327, serial currently committed 0x035f1357051c7199 2005-08-03 12:39:01.197916)
On 2005-08-04, at 09:02 GMT+12:00, Maciej Wisniowski wrote:
so I hope that maybe next Zope 2.8.x release will be using MVCC for OOBTree (if this is the point?). Does anybody know something about that?
That looks like a write conflict. MVCC helps to lower/eliminate the rate of read conflicts. One option to reduce the rate of this write conflict is to tune the session machinery to suit. For example use session-resolution-seconds of say 300 seconds. michael
That looks like a write conflict. MVCC helps to lower/eliminate the rate of read conflicts. One option to reduce the rate of this write conflict is to tune the session machinery to suit. For example use session-resolution-seconds of say 300 seconds. Thanks! I've changed session-resolution-seconds and did some tests again. With DCOracle 2 there are still some errors (30 conflicts visible for users for 3000 requests), but with SQLRelay there are no errors at all! :) I didn't checked all 30 errors but I think these are all conflicts errors with Products.DCOracle2.DA.Procedure object like this one:
<P>An error was encountered while publishing this resource. </P> <P><STRONG>ZODB.POSException.ConflictError</STRONG></P> Sorry, a site error occurred.<p><p>Traceback (innermost last): <ul> <li> Module ZPublisher.Publish, line 187, in publish_module_standard</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 151, in publish</li> <li> Module Zope2.App.startup, line 158, in zpublisher_exception_hook</li> <li> Module ZPublisher.Publish, line 119, in publish</li> <li> Module Zope2.App.startup, line 215, in commit</li> <li> Module transaction._manager, line 84, in commit</li> <li> Module transaction._transaction, line 381, in commit</li> <li> Module transaction._transaction, line 379, in commit</li> <li> Module transaction._transaction, line 427, in _commitResources</li> <li> Module ZODB.Connection, line 637, in tpc_vote</li> <li> Module ZEO.ClientStorage, line 893, in tpc_vote</li> <li> Module ZEO.ClientStorage, line 877, in _check_serials</li> </ul>ConflictError: database conflict error (oid 0x0299de, class Products.DCOracle2.DA.Procedure, serial this txn started with 0x035f181b76214155 2005-08-04 08:59:27.686696, serial currently committed 0x035f181b79752811 2005-08-04 08:59:28.466635) Any ideas what can be done to remove this problem, except the obvious solution - that is moving to SQLRelay ;) I tried _p_resolveConflict function but without effect - it's used to resolve write conflict and I suppose these errors like the one above are read conflicts. -- Maciej Wisniowki
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Maciej Wisniowski wrote:
That looks like a write conflict. MVCC helps to lower/eliminate the rate of read conflicts. One option to reduce the rate of this writeconflict is to tune the session machinery to suit. For example use session-resolution-seconds of say 300 seconds.
Thanks! I've changed session-resolution-seconds and did some tests again. With DCOracle 2 there are still some errors (30 conflicts visible for users for 3000 requests), but with SQLRelay there are no errors at all! :) I didn't checked all 30 errors but I think these are all conflicts errors with Products.DCOracle2.DA.Procedure object like this one:
<P>An error was encountered while publishing this resource. </P> <P><STRONG>ZODB.POSException.ConflictError</STRONG></P> Sorry, a site error occurred.<p><p>Traceback (innermost last): <ul> <li> Module ZPublisher.Publish, line 187, in publish_module_standard</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 151, in publish</li> <li> Module Zope2.App.startup, line 158, in zpublisher_exception_hook</li> <li> Module ZPublisher.Publish, line 119, in publish</li> <li> Module Zope2.App.startup, line 215, in commit</li> <li> Module transaction._manager, line 84, in commit</li> <li> Module transaction._transaction, line 381, in commit</li> <li> Module transaction._transaction, line 379, in commit</li> <li> Module transaction._transaction, line 427, in _commitResources</li> <li> Module ZODB.Connection, line 637, in tpc_vote</li> <li> Module ZEO.ClientStorage, line 893, in tpc_vote</li> <li> Module ZEO.ClientStorage, line 877, in _check_serials</li> </ul>ConflictError: database conflict error (oid 0x0299de, class Products.DCOracle2.DA.Procedure, serial this txn started with 0x035f181b76214155 2005-08-04 08:59:27.686696, serial currently committed 0x035f181b79752811 2005-08-04 08:59:28.466635)
Any ideas what can be done to remove this problem, except the obvious solution - that is moving to SQLRelay ;) I tried _p_resolveConflict function but without effect - it's used to resolve write conflict and I suppose these errors like the one above are read conflicts.
Nope; any conflict propagated from within 'commit' is going to be a write conflict; more than one transaction is writing to the same persistent object (in the ZODB) at the same time. Tres. - -- =================================================================== Tres Seaver +1 202-558-7113 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFC81Vd+gerLs4ltQ4RAtaEAKDAokv4+azXlqfF2vppOCnhmp2tTwCgv+uw v9w7biMr71rQLFtynCZN0wU= =gzYB -----END PGP SIGNATURE-----
Maciej Wisniowski wrote at 2005-8-5 09:23 +0200:
... <li> Module ZEO.ClientStorage, line 877, in _check_serials</li> </ul>ConflictError: database conflict error (oid 0x0299de, class Products.DCOracle2.DA.Procedure, serial this txn started with 0x035f181b76214155 2005-08-04 08:59:27.686696, serial currently committed 0x035f181b79752811 2005-08-04 08:59:28.466635)
Do you see a reason why your requests modify "Procedure" instances? I find this strange: I would expect that they are rarely modified (just once to set them up). -- Dieter
Do you see a reason why your requests modify "Procedure" instances? I find this strange: I would expect that they are rarely modified (just once to set them up). It's possibly because of _connect method in DCOracle2 SP.py. It assigns to self _v_proc, connection, and errors count.
Yes, it's a bit strange and possibly unnecessary. I've already looked at this but had no time to try doing changes in DCO2 code - I stared testing other branch of DCO2 from its CVS and SQLRelay. DCOracle2 is far more complicated than SQLRelayDA or PsycopgDA. I've crated my own OracleStoredProcedure product for SQLRelay which behaves just like Procedure from DCO2 but has a bit more complicated usage. It's written in far more simple way and thanks to this more stable I think :) Thanks to pointing this, I'll try to take closer look. Maybe assigning self._v_proc during initialization will do the thing... but I'm afraid it's not so simple ;) One more question - am I correct that _p_resolveConflict is not used anymore in Zope 2.8.x? -- Maciej Wisniowski
[Maciej Wisniowski]
... One more question - am I correct that _p_resolveConflict is not used anymore in Zope 2.8.x?
Not correct. _p_resolveConflict is used in Zope 2.8.x (and in Zope 3) exactly the same way it was used in 2.7.x. If it's not working, there may be a new bug in 2.8.x, but there's been report of that yet.
Not correct. _p_resolveConflict is used in Zope 2.8.x (and in Zope 3)
exactly the same way it was used in 2.7.x. If it's not working, there may be a new bug in 2.8.x, but there's been report of that yet.
Because of errors like: ConflictError: database conflict error (oid 0x0299de, class Products.DCOracle2.DA.Procedure, serial this txn started with 0x035f0726c7bfca88 2005-08-01 08:38:46.816215, serial currently committed 0x035f072fed78ebee 2005-08-01 08:47:55.657582) and according to: http://www.upfrontsystems.co.za/Members/jean/conflicterrors-explained I've added: def _p_resolveConflict(self, old, saved, new): print 'old:', old print 'saved:', saved print 'new:', new return new to SP.py into procedure class (DA.Procedure extends this), I've started Zope with runzope and there are no 'old':?, 'new':?, 'saved:?' statements at all on the console although other print statements from SP.Procedure are visible. Maybe these conflict errors I'm dealing with are different kind and are not falling into the correct piece of code or maybe this kind of test (print 'something') is too stupid to see if _p_resolveConflict is executed? -- Maciej Wisniowski
[Maciej Wisniowski, wrestling with _p_resolveConflict]
Because of errors like:
ConflictError: database conflict error (oid 0x0299de, class Products.DCOracle2.DA.Procedure, serial this txn started with 0x035f0726c7bfca88 2005-08-01 08:38:46.816215, serial currently committed 0x035f072fed78ebee 2005-08-01 08:47:55.657582)
and according to: http://www.upfrontsystems.co.za/Members/jean/conflicterrors-explained I've added:
def _p_resolveConflict(self, old, saved, new): print 'old:', old print 'saved:', saved print 'new:', new return new
to SP.py into procedure class (DA.Procedure extends this),
I've started Zope with runzope and there are no 'old':?, 'new':?, 'saved:?' statements at all on the console although other print statements from SP.Procedure are visible.
Maybe these conflict errors I'm dealing with are different kind and are not falling into the correct piece of code or maybe this kind of test (print 'something') is too stupid to see if _p_resolveConflict is executed?
Offhand it looks OK to me, but I don't have any of the code you're using. If you want to dig deeper, look into what happens inside lib/python/ZODB/ConflictResolution.py in function tryToResolveConflict(). If your _p_resolveConflict() were getting called, you would see the print output you coded. Therefore it's not getting called. tryToResolveConflict() is what _tries_ to call it (assuming that for an instance `object` of this type, getattr(object, "_p_resolveConflict", None) returns a non-None value to begin with, which I can't tell for sure from what you've said).
Offhand it looks OK to me, but I don't have any of the code you're using.
I think that it's not my code problem but rather something with DCOracle2 and to reproduce this error it is enough to call any 'DCOracle2 StoredProcedure' object simultaneously by a lot of users (50 is enough for my Zope (ZEO) to cause these errors).
If you want to dig deeper, look into what happens inside lib/python/ZODB/ConflictResolution.py
I'll try to check this, thanks for the hint. :) -- Maciej Wisniowski
[Tim Peters]
Offhand it looks OK to me, but I don't have any of the code you're using.
[Maciej Wisniowski]
I think that it's not my code problem but rather something with DCOracle2 and to reproduce this error it is enough to call any 'DCOracle2 StoredProcedure' object simultaneously by a lot of users (50 is enough for my Zope (ZEO) to cause these errors).
Sorry, I'm not making myself clear: I don't have _any_ of the code you're using, including DCOracle2. I don't use it -- and I don't have time to try to figure out how to use it. But I know the code paths in ZODB on the way toward calling _p_resolveConflict, and if I did have the code you were using, this is what I'd do:
If you want to dig deeper, look into what happens inside lib/python/ZODB/ConflictResolution.py
I'll try to check this, thanks for the hint. :)
It's the only way you'll figure it out ;-)
Well, you know... In this one project, we had an interesting (cough) situation with PostgreSQL + psycopg, where we would see plenty of write conflict errors coming from the DB connector. Turned out PostgreSQL has kind of an "optimistic" transaction control, where the DB server returns something like "couldn't really resolve this concurrency issue, please retry" to the DB connector, and psycopg would translate that into a ZODB ConflictError to make the ZPublisher retry the transaction. Not kidding. So, while I don't know much about the internals of DCOracle you may want to make sure the ConflictErrors are due to the ZODB at all. Stefan On 7. Aug 2005, at 23:50, Tim Peters wrote:
If you want to dig deeper, look into what happens inside lib/python/ZODB/ConflictResolution.py
I'll try to check this, thanks for the hint. :)
It's the only way you'll figure it out ;-)
-- Anything that happens, happens. --Douglas Adams
Maciej Wisniowski wrote at 2005-8-5 23:37 +0200:
Do you see a reason why your requests modify "Procedure" instances? I find this strange: I would expect that they are rarely modified (just once to set them up). It's possibly because of _connect method in DCOracle2 SP.py. It assigns to self _v_proc, connection, and errors count.
Assignments to "_v_" attributes (they are so called "volatile" attributes) do not change the persistent state and cannot cause conflicts. There must be some other modification to the procedure objects. -- Dieter
Assignments to "_v_" attributes (they are so called "volatile" attributes) do not change the persistent state and cannot cause conflicts.
There must be some other modification to the procedure objects.
I should have written: "It's possibly because of _connect method in DCOracle2 SP.py. It assigns values to self._v_proc, self.connection and self.errors." So except '_v_proc' there are 'connection' and 'errors' variables that are being assigned to. Maybe this it's because of them. -- Maciej Wisniowski
Maciej Wisniowski wrote at 2005-8-1 12:21 +0200:
... Other errors are like "Shouldn't load state xxx when connection is closed" but these are known bugs in older ZODB version.
Really? I do not know such errors in any ZODB version. I know these errors as persistency bugs in applications (among others some of mine)... -- Dieter
Other errors are like "Shouldn't load state xxx when connection is closed" but these are known bugs in older ZODB version.
Really? I do not know such errors in any ZODB version.
I know these errors as persistency bugs in applications (among others some of mine)...
I've found this information here: http://plone.org/documentation/error/loadstate ?? -- Maciej Wisniowski
participants (8)
-
Chris McDonough -
Dieter Maurer -
Maciej Wisniowski -
Michael Dunstan -
Paul Winkler -
Stefan H. Holek -
Tim Peters -
Tres Seaver