[Zope] Zope 2.8 and conflict errors
Chris McDonough
chrism at plope.com
Mon Aug 1 10:55:04 EDT 2005
Hi Maciej,
I can't speak to the 500 errors you're receiving but Zope 2.8.0 ships
with a "TemporaryStorage" implementation (used by sessions in the
default configuration) which does not implement MVCC. The one in 2.8.1
(now in beta) and better will implement MVCC, so you *should* see
performance improve (or at least see fewer conflict errors while using
sessioning). I'm keenly interested in your results, so please let me
know.
- C
On Mon, 2005-08-01 at 12:21 +0200, Maciej Wisniowski wrote:
> Hi!
>
> Does anybody tried 2.8 and can say something about conflict errors?
>
> I've done few tests and results are not good.
>
> We've checked one of our applications on Zope 2.7.2 and the same
> application on Zope 2.8. We have simulated 50-65 users executing the
> same path in application. There was total count 1500 pages visited
> during test.
>
> With Zope 2.7.2 we've received about 39 internal server errors
> HTTP/1.1 500.
>
> Errors in Zope 2.7.2 are mainly ReadConflictErrors with
> DCOracle2.DA.Procedure like:
>
> ------
> 2005-08-01T09:54:08 INFO(0) TM:65540 Failed to abort object
> Traceback (most recent call last):
> File "/opt/Zope/2.7.2/lib/python/ZODB/Transaction.py", line 393, in
> _commit_error
> j.abort(o, self)
> File "/opt/Zope/2.7.2/lib/python/ZODB/Connection.py", line 561, in
> setstate
> invalid = self._is_invalidated(obj)
> File "/opt/Zope/2.7.2/lib/python/ZODB/Connection.py", line 590, in
> _is_invalidated
> raise ReadConflictError(object=obj)
> ReadConflictError: database read conflict error (oid 00000000000299de,
> class Products.DCOracle2.DA.Procedure)
> ------
>
> Other errors are like "Shouldn't load state xxx when
> connection is closed" but these are known bugs in older ZODB version.
>
>
> Zope 2.8 supports MVCC so we hoped that the number of
> errors decrease but... with Zope 2.8 we get about 129 internal
> server errors HTTP/1.1 500 and additionally a lot of 402 errors
> (unauthorized).
>
> Errors 402 are because of internal server errors on login page
> (testing application continues test without checking if user has been
> properly logged in).
>
> In the event.log there are no conflict errors saved :-/ ,
> only unauthorized. We get the error from client responses
> and from console.
>
> I don't know what happens. May it be because of using old
> data.fs file (from 2.7.2) with Zope 2.8?
>
> Anybody has similiar experiences? Is Zope 2.8 useable??
>
>
> Below few errors from the console (runzope) and from user browser:
>
> ----------------------------------------------------------------
> 2005-08-01 10:46:22 INFO ZODB conflict error at /crProd/acl_users/login
> (16 conflicts since startup at 2005-08-01T10:45:58)
> 2005-08-01 10:46:22 BLATHER Conflict traceback
> Traceback (most recent call last):
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in
> publish
> request, bind=1)
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply
> if debug is not None: return debug(object,args,context)
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in
> call_object
> result=apply(object,args) # Type s<cr> to step into published object.
> File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line
> 311, in __call__
> return self._bindAndExec(args, kw, None)
> File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line
> 348, in _bindAndExec
> return self._exec(bound_data, args, kw)
> File
> "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py",
> line 323, in _exec
> result = f(*args, **kw)
> File "Script (Python)", line 3, in login
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line
> 1214, in __getattr__
> v = self.get(key, default, returnTaints=returnTaints)
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line
> 1174, in get
> if callable(v): v = v()
> File
> "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py",
> line 93, in getSessionData
> return self._getSessionDataObject(key)
> File
> "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py",
> line 180, in _getSessionDataObject
> ob = container.new_or_existing(key)
> File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py",
> line 844, in new_or_existing
> item = self.get(key, _marker)
> File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py",
> line 491, in get
> item = self._move_item(k, current_ts, default)
> File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py",
> line 297, in _move_item
> abucket = self._data.get(ts, None) # XXX ReadConflictError hotspot
> File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in
> setstate
> self._setstate(obj)
> File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in
> _setstate
> self._load_before_or_conflict(obj)
> File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in
> _load_before_or_conflict
> raise ReadConflictError(object=obj)
> ReadConflictError: database read conflict error (oid 0x0a, class
> BTrees._IOBTree.IOBTree)
> ----------------------------------------------------------------
>
> error in /crProd/acl_users/login - is with request.SESSION statement.
>
> ----------------------------------------------------------------
> 2005-08-01 10:46:21 BLATHER Conflict traceback
> Traceback (most recent call last):
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in
> publish
> request, bind=1)
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply
> if debug is not None: return debug(object,args,context)
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in
> call_object
> result=apply(object,args) # Type s<cr> to step into published object.
> File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line
> 311, in __call__
> return self._bindAndExec(args, kw, None)
> File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line
> 348, in _bindAndExec
> return self._exec(bound_data, args, kw)
> File
> "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py",
> line 323, in _exec
> result = f(*args, **kw)
> File "Script (Python)", line 3, in login
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line
> 1214, in __getattr__
> v = self.get(key, default, returnTaints=returnTaints)
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line
> 1174, in get
> if callable(v): v = v()
> File
> "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py",
> line 93, in getSessionData
> return self._getSessionDataObject(key)
> File
> "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py",
> line 180, in _getSessionDataObject
> ob = container.new_or_existing(key)
> File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py",
> line 844, in new_or_existing
> item = self.get(key, _marker)
> File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py",
> line 491, in get
> item = self._move_item(k, current_ts, default)
> File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py",
> line 280, in _move_item
> self._housekeep(current_ts)
> File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py",
> line 939, in _housekeep
> self._finalize(now)
> File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py",
> line 547, in _finalize
> last_finalized = self._last_finalized_timeslice()
> File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py",
> line 1076, in __call__
> return self.value
> File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in
> setstate
> self._setstate(obj)
> File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in
> _setstate
> self._load_before_or_conflict(obj)
> File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in
> _load_before_or_conflict
> raise ReadConflictError(object=obj)
> ReadConflictError: database read conflict error (oid 0x09, class
> Products.Transience.Transience.Increaser)
> -----------------------------------------------------------
>
> or
>
> -----------------------------------------------------------
> 2005-08-01 10:46:21 BLATHER Conflict traceback
> Traceback (most recent call last):
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in
> publish
> request, bind=1)
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply
> if debug is not None: return debug(object,args,context)
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in
> call_object
> result=apply(object,args) # Type s<cr> to step into published object.
> File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line
> 311, in __call__
> return self._bindAndExec(args, kw, None)
> File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line
> 348, in _bindAndExec
> return self._exec(bound_data, args, kw)
> File
> "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py",
> line 323, in _exec
> result = f(*args, **kw)
> File "Script (Python)", line 3, in login
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line
> 1214, in __getattr__
> v = self.get(key, default, returnTaints=returnTaints)
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line
> 1174, in get
> if callable(v): v = v()
> File
> "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py",
> line 93, in getSessionData
> return self._getSessionDataObject(key)
> File
> "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py",
> line 180, in _getSessionDataObject
> ob = container.new_or_existing(key)
> File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py",
> line 847, in new_or_existing
> self[key] = item
> File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py",
> line 435, in __setitem__
> length = self._length() # XXX ReadConflictError hotspot
> File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py",
> line 1119, in __call__
> return self.value
> File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in
> setstate
> self._setstate(obj)
> File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in
> _setstate
> self._load_before_or_conflict(obj)
> File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in
> _load_before_or_conflict
> raise ReadConflictError(object=obj)
> ReadConflictError: database read conflict error (oid 0x08, class
> Products.Transience.Transience.Length2)
> -----------------------------------------------------------
>
> or
>
> -----------------------------------------------------------
> 2005-08-01 10:46:21 BLATHER Conflict traceback
> Traceback (most recent call last):
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 113, in
> publish
> request, bind=1)
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/mapply.py", line 88, in mapply
> if debug is not None: return debug(object,args,context)
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 40, in
> call_object
> result=apply(object,args) # Type s<cr> to step into published object.
> File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line
> 311, in __call__
> return self._bindAndExec(args, kw, None)
> File "/opt/Zope/2.8.0/lib/python/Shared/DC/Scripts/Bindings.py", line
> 348, in _bindAndExec
> return self._exec(bound_data, args, kw)
> File
> "/opt/Zope/2.8.0/lib/python/Products/PythonScripts/PythonScript.py",
> line 323, in _exec
> result = f(*args, **kw)
> File "Script (Python)", line 3, in login
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line
> 1214, in __getattr__
> v = self.get(key, default, returnTaints=returnTaints)
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/HTTPRequest.py", line
> 1174, in get
> if callable(v): v = v()
> File
> "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py",
> line 93, in getSessionData
> return self._getSessionDataObject(key)
> File
> "/opt/Zope/2.8.0/lib/python/Products/Sessions/SessionDataManager.py",
> line 180, in _getSessionDataObject
> ob = container.new_or_existing(key)
> File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py",
> line 844, in new_or_existing
> item = self.get(key, _marker)
> File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py",
> line 491, in get
> item = self._move_item(k, current_ts, default)
> File "/opt/Zope/2.8.0/lib/python/Products/Transience/Transience.py",
> line 297, in _move_item
> abucket = self._data.get(ts, None) # XXX ReadConflictError hotspot
> File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 704, in
> setstate
> self._setstate(obj)
> File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 757, in
> _setstate
> self._load_before_or_conflict(obj)
> File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 768, in
> _load_before_or_conflict
> raise ReadConflictError(object=obj)
> ReadConflictError: database read conflict error (oid 0x0a, class
> BTrees._IOBTree.IOBTree)
> -----------------------------------------------------------
>
> or
>
> -----------------------------------------------------------
> 2005-08-01 10:47:55 INFO ZODB conflict error at
> /crProd/cr/centralne/procName (87 conflicts since startup at
> 2005-08-01T10:45:58)
> 2005-08-01 10:47:55 BLATHER Conflict traceback
> Traceback (most recent call last):
> File "/opt/Zope/2.8.0/lib/python/ZPublisher/Publish.py", line 119, in
> publish
> transactions_manager.commit()
> File "/opt/Zope/2.8.0/lib/python/Zope2/App/startup.py", line 215, in
> commit
> transaction.commit()
> File "/opt/Zope/2.8.0/lib/python/transaction/_manager.py", line 84, in
> commit
> self.get().commit(sub)
> File "/opt/Zope/2.8.0/lib/python/transaction/_transaction.py", line
> 382, in commit
> self._saveCommitishError() # This raises!
> File "/opt/Zope/2.8.0/lib/python/transaction/_transaction.py", line
> 380, in commit
> self._commitResources()
> File "/opt/Zope/2.8.0/lib/python/transaction/_transaction.py", line
> 428, in _commitResources
> rm.tpc_vote(self)
> File "/opt/Zope/2.8.0/lib/python/ZODB/Connection.py", line 637, in
> tpc_vote
> s = vote(transaction)
> File "/opt/Zope/2.8.0/lib/python/ZEO/ClientStorage.py", line 893, in
> tpc_vote
> return self._check_serials()
> File "/opt/Zope/2.8.0/lib/python/ZEO/ClientStorage.py", line 877, in
> _check_serials
> raise s
> ConflictError: database conflict error (oid 0x0299de, class
> Products.DCOracle2.DA.Procedure, serial this txn started with
> 0x035f0726c7bfca88 2005-08-01 08:38:46.816215, serial currently
> committed 0x035f072fed78ebee 2005-08-01 08:47:55.657582)
> -----------------------------------------------------------
>
> User receives errors like:
> -----------------------------------------------------------------
> Response:
> HTTP/1.1 500 Internal Server Error
> Server: Zope/(Zope 2.8.0-final, python 2.3.5, linux2) ZServer/1.1
> Date: Fri, 29 Jul 2005 12:53:00 GMT
> Bobo-Exception-Line: 877
> Content-Length: 2020
> Bobo-Exception-Value: See the server error log for details
> Content-Language: pl
> Bobo-Exception-File: ClientStorage.py
> Bobo-Exception-Type: ZODB.POSException.ConflictError
>
> (...)
>
> <TD WIDTH="90%">
> <H2>Site Error</H2>
> <P>An error was encountered while publishing this resource.
> </P>
> <P><STRONG>ZODB.POSException.ConflictError</STRONG></P>
> Sorry, a site error occurred.<p><p>Traceback (innermost last):
> <ul>
>
> Module ZPublisher.Publish, line 187, in publish_module_standard
> <li> Module ZPublisher.Publish, line 161, in publish</li>
> <li> Module ZPublisher.Publish, line 161, in publish</li>
> <li> Module ZPublisher.Publish, line 161, in publish</li>
> <li> Module ZPublisher.Publish, line 151, in publish</li>
> <li> Module Zope2.App.startup, line 158, in zpublisher_exception_hook</li>
> <li> Module ZPublisher.Publish, line 119, in publish</li>
> <li> Module Zope2.App.startup, line 215, in commit</li>
> <li> Module transaction._manager, line 84, in commit</li>
> <li> Module transaction._transaction, line 382, in commit</li>
> <li> Module transaction._transaction, line 380, in commit</li>
> <li> Module transaction._transaction, line 428, in _commitResources</li>
> <li> Module ZODB.Connection, line 637, in tpc_vote</li>
> <li> Module ZEO.ClientStorage, line 893, in tpc_vote</li>
> <li> Module ZEO.ClientStorage, line 877, in _check_serials</li>
> </ul>ConflictError: database conflict error (oid 0x0299de, class
> Products.DCOracle2.DA.Procedure, serial this txn started with
> 0x035ef744f3d9b366 2005-07-29 12:52:57.152436, serial currently
> committed 0x035ef744fe4886cc 2005-07-29 12:52:59.597650)
> </p>
> <HR NOSHADE>
> <P>Troubleshooting Suggestions</P>
> (...)
> -----------------------------------------------------------------
>
>
More information about the Zope
mailing list