Hi Maciej, I can't speak to the 500 errors you're receiving but Zope 2.8.0 ships with a "TemporaryStorage" implementation (used by sessions in the default configuration) which does not implement MVCC. The one in 2.8.1 (now in beta) and better will implement MVCC, so you *should* see performance improve (or at least see fewer conflict errors while using sessioning). I'm keenly interested in your results, so please let me know. - C On Mon, 2005-08-01 at 12:21 +0200, Maciej Wisniowski wrote:
Hi!
Does anybody tried 2.8 and can say something about conflict errors?
I've done few tests and results are not good.
We've checked one of our applications on Zope 2.7.2 and the same application on Zope 2.8. We have simulated 50-65 users executing the same path in application. There was total count 1500 pages visited during test.
With Zope 2.7.2 we've received about 39 internal server errors HTTP/1.1 500.
Errors in Zope 2.7.2 are mainly ReadConflictErrors with DCOracle2.DA.Procedure like:
------ 2005-08-01T09:54:08 INFO(0) TM:65540 Failed to abort object Traceback (most recent call last): File "/opt/Zope/2.7.2/lib/python/ZODB/Transaction.py", line 393, in _commit_error j.abort(o, self) File "/opt/Zope/2.7.2/lib/python/ZODB/Connection.py", line 561, in setstate invalid = self._is_invalidated(obj) File "/opt/Zope/2.7.2/lib/python/ZODB/Connection.py", line 590, in _is_invalidated raise ReadConflictError(object=obj) ReadConflictError: database read conflict error (oid 00000000000299de, class Products.DCOracle2.DA.Procedure) ------
Other errors are like "Shouldn't load state xxx when connection is closed" but these are known bugs in older ZODB version.
Zope 2.8 supports MVCC so we hoped that the number of errors decrease but... with Zope 2.8 we get about 129 internal server errors HTTP/1.1 500 and additionally a lot of 402 errors (unauthorized).
Errors 402 are because of internal server errors on login page (testing application continues test without checking if user has been properly logged in).
In the event.log there are no conflict errors saved :-/ , only unauthorized. We get the error from client responses and from console.
I don't know what happens. May it be because of using old data.fs file (from 2.7.2) with Zope 2.8?
Anybody has similiar experiences? Is Zope 2.8 useable??
Below few errors from the console (runzope) and from user browser:
---------------------------------------------------------------- 2005-08-01 10:46:22 INFO ZODB conflict error at /crProd/acl_users/login (16 conflicts since startup at 2005-08-01T10:45:58) 2005-08-01 10:46:22 BLATHER Conflict traceback Traceback (most recent call last): File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in publish request, bind=1) File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply if debug is not None: return debug(object,args,context) File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in call_object result=apply(object,args) # Type s<cr> to step into published object. File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 311, in __call__ return self._bindAndExec(args, kw, None) File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 348, in _bindAndExec return self._exec(bound_data, args, kw) File "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", line 323, in _exec result = f(*args, **kw) File "Script (Python)", line 3, in login File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1214, in __getattr__ v = self.get(key, default, returnTaints=returnTaints) File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1174, in get if callable(v): v = v() File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 93, in getSessionData return self._getSessionDataObject(key) File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 180, in _getSessionDataObject ob = container.new_or_existing(key) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 844, in new_or_existing item = self.get(key, _marker) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 491, in get item = self._move_item(k, current_ts, default) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 297, in _move_item abucket = self._data.get(ts, None) # XXX ReadConflictError hotspot File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in setstate self._setstate(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in _setstate self._load_before_or_conflict(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in _load_before_or_conflict raise ReadConflictError(object=obj) ReadConflictError: database read conflict error (oid 0x0a, class BTrees._IOBTree.IOBTree) ----------------------------------------------------------------
error in /crProd/acl_users/login - is with request.SESSION statement.
---------------------------------------------------------------- 2005-08-01 10:46:21 BLATHER Conflict traceback Traceback (most recent call last): File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in publish request, bind=1) File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply if debug is not None: return debug(object,args,context) File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in call_object result=apply(object,args) # Type s<cr> to step into published object. File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 311, in __call__ return self._bindAndExec(args, kw, None) File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 348, in _bindAndExec return self._exec(bound_data, args, kw) File "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", line 323, in _exec result = f(*args, **kw) File "Script (Python)", line 3, in login File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1214, in __getattr__ v = self.get(key, default, returnTaints=returnTaints) File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1174, in get if callable(v): v = v() File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 93, in getSessionData return self._getSessionDataObject(key) File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 180, in _getSessionDataObject ob = container.new_or_existing(key) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 844, in new_or_existing item = self.get(key, _marker) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 491, in get item = self._move_item(k, current_ts, default) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 280, in _move_item self._housekeep(current_ts) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 939, in _housekeep self._finalize(now) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 547, in _finalize last_finalized = self._last_finalized_timeslice() File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 1076, in __call__ return self.value File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in setstate self._setstate(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in _setstate self._load_before_or_conflict(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in _load_before_or_conflict raise ReadConflictError(object=obj) ReadConflictError: database read conflict error (oid 0x09, class Products.Transience.Transience.Increaser) -----------------------------------------------------------
or
----------------------------------------------------------- 2005-08-01 10:46:21 BLATHER Conflict traceback Traceback (most recent call last): File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in publish request, bind=1) File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply if debug is not None: return debug(object,args,context) File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in call_object result=apply(object,args) # Type s<cr> to step into published object. File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 311, in __call__ return self._bindAndExec(args, kw, None) File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 348, in _bindAndExec return self._exec(bound_data, args, kw) File "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", line 323, in _exec result = f(*args, **kw) File "Script (Python)", line 3, in login File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1214, in __getattr__ v = self.get(key, default, returnTaints=returnTaints) File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1174, in get if callable(v): v = v() File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 93, in getSessionData return self._getSessionDataObject(key) File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 180, in _getSessionDataObject ob = container.new_or_existing(key) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 847, in new_or_existing self[key] = item File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 435, in __setitem__ length = self._length() # XXX ReadConflictError hotspot File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 1119, in __call__ return self.value File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in setstate self._setstate(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in _setstate self._load_before_or_conflict(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in _load_before_or_conflict raise ReadConflictError(object=obj) ReadConflictError: database read conflict error (oid 0x08, class Products.Transience.Transience.Length2) -----------------------------------------------------------
or
----------------------------------------------------------- 2005-08-01 10:46:21 BLATHER Conflict traceback Traceback (most recent call last): File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in publish request, bind=1) File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply if debug is not None: return debug(object,args,context) File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in call_object result=apply(object,args) # Type s<cr> to step into published object. File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 311, in __call__ return self._bindAndExec(args, kw, None) File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line 348, in _bindAndExec return self._exec(bound_data, args, kw) File "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py", line 323, in _exec result = f(*args, **kw) File "Script (Python)", line 3, in login File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1214, in __getattr__ v = self.get(key, default, returnTaints=returnTaints) File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line 1174, in get if callable(v): v = v() File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 93, in getSessionData return self._getSessionDataObject(key) File "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py", line 180, in _getSessionDataObject ob = container.new_or_existing(key) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 844, in new_or_existing item = self.get(key, _marker) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 491, in get item = self._move_item(k, current_ts, default) File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py", line 297, in _move_item abucket = self._data.get(ts, None) # XXX ReadConflictError hotspot File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in setstate self._setstate(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in _setstate self._load_before_or_conflict(obj) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in _load_before_or_conflict raise ReadConflictError(object=obj) ReadConflictError: database read conflict error (oid 0x0a, class BTrees._IOBTree.IOBTree) -----------------------------------------------------------
or
----------------------------------------------------------- 2005-08-01 10:47:55 INFO ZODB conflict error at /crProd/cr/centralne/procName (87 conflicts since startup at 2005-08-01T10:45:58) 2005-08-01 10:47:55 BLATHER Conflict traceback Traceback (most recent call last): File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 119, in publish transactions_manager.commit() File "/opt/Zope/2.8.0/lib/python/Zope2/App/startup.py", line 215, in commit transaction.commit() File "/opt/Zope/2.8.0/lib/python/transaction/_manager.py", line 84, in commit self.get().commit(sub) File "/opt/Zope/2.8.0/lib/python/transaction/_transaction.py", line 382, in commit self._saveCommitishError() # This raises! File "/opt/Zope/2.8.0/lib/python/transaction/_transaction.py", line 380, in commit self._commitResources() File "/opt/Zope/2.8.0/lib/python/transaction/_transaction.py", line 428, in _commitResources rm.tpc_vote(self) File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 637, in tpc_vote s = vote(transaction) File "/opt/Zope/2.8.0/lib/python/ZEO/ClientStorage.py", line 893, in tpc_vote return self._check_serials() File "/opt/Zope/2.8.0/lib/python/ZEO/ClientStorage.py", line 877, in _check_serials raise s ConflictError: database conflict error (oid 0x0299de, class Products.DCOracle2.DA.Procedure, serial this txn started with 0x035f0726c7bfca88 2005-08-01 08:38:46.816215, serial currently committed 0x035f072fed78ebee 2005-08-01 08:47:55.657582) -----------------------------------------------------------
User receives errors like: ----------------------------------------------------------------- Response: HTTP/1.1 500 Internal Server Error Server: Zope/(Zope 2.8.0-final, python 2.3.5, linux2) ZServer/1.1 Date: Fri, 29 Jul 2005 12:53:00 GMT Bobo-Exception-Line: 877 Content-Length: 2020 Bobo-Exception-Value: See the server error log for details Content-Language: pl Bobo-Exception-File: ClientStorage.py Bobo-Exception-Type: ZODB.POSException.ConflictError
(...)
<TD WIDTH="90%"> <H2>Site Error</H2> <P>An error was encountered while publishing this resource. </P> <P><STRONG>ZODB.POSException.ConflictError</STRONG></P> Sorry, a site error occurred.<p><p>Traceback (innermost last): <ul>
Module ZPublisher.Publish, line 187, in publish_module_standard <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 161, in publish</li> <li> Module ZPublisher.Publish, line 151, in publish</li> <li> Module Zope2.App.startup, line 158, in zpublisher_exception_hook</li> <li> Module ZPublisher.Publish, line 119, in publish</li> <li> Module Zope2.App.startup, line 215, in commit</li> <li> Module transaction._manager, line 84, in commit</li> <li> Module transaction._transaction, line 382, in commit</li> <li> Module transaction._transaction, line 380, in commit</li> <li> Module transaction._transaction, line 428, in _commitResources</li> <li> Module ZODB.Connection, line 637, in tpc_vote</li> <li> Module ZEO.ClientStorage, line 893, in tpc_vote</li> <li> Module ZEO.ClientStorage, line 877, in _check_serials</li> </ul>ConflictError: database conflict error (oid 0x0299de, class Products.DCOracle2.DA.Procedure, serial this txn started with 0x035ef744f3d9b366 2005-07-29 12:52:57.152436, serial currently committed 0x035ef744fe4886cc 2005-07-29 12:52:59.597650) </p> <HR NOSHADE> <P>Troubleshooting Suggestions</P> (...) -----------------------------------------------------------------