[ZCM] [ZC] 869/ 6 Reject "Broken transaction handling in case of exceptions"

Sun May 23 02:00:16 EDT 2004

Issue #869 Update (Reject) "Broken transaction handling in case of exceptions"
 Status Rejected, Zope/bug medium
To followup, visit:
  http://collector.zope.org/Zope/869

==============================================================
= Reject - Entry #6 by mcdonc on May 23, 2004 2:00 am

 Status: Pending => Rejected

This is a duplicate of 789, so I am "rejecting" it (although the issue is valid, and has been fixed for 2.7.1 and HEAD, see issue 789).
________________________________________
= Comment - Entry #5 by d.maurer on Apr 4, 2003 2:29 am

I am not in line with your principle "the error report is
part of [the output of] the first transaction". My intuition
here comes from more conventional databases where a
transaction can be aborted at any time by the database system.
But, even with Zope, when the error report should do
something persistently, it must do it in a new transaction
as the old state is inconsistent and can not be committed.

Your scenario correctly points out a problem to handled
with the error report in a different transaction.
However, the problem seems not to occur with high probability.

With respect to "zodb-level" badness:
When the persistent objects referenced in the traceback
have been modified, then they will be invalidated by the
preceeding "abort" (drawback: you will not see the (inconsistent) state created by erroneous transaction
but a fresh copy). When they have not been modified,
you have the intrinsic ZODB problem that it allows
to read stale data from the cache. Because of an intermediate
transaction boundary (error report in its own transaction),
the probability of reading stale data is reduced.
Drawback: you may see newer data in the error report than
has been seen by the erroneous transaction.

________________________________________
= Comment - Entry #4 by htrd on Mar 27, 2003 2:58 pm

I can see many problems with doing error reporting in a new transaction. As a matter of principal, the error report is part of the output of that first transaction. How can we generate a sensible application-level error page when the error occurred in a different transaction, and the application state may have changed arbitrarily? Surely we need atomicity between the failing operation and its subsequent error report.

Consider the following scenario:
1. request A causes an exception in object M
2. a concurrent transaction deletes M, changes its state, or inteferes in some other way.
3. to finish handling request A we have to generate an error page for the now non-existant object M.

I can also see some zodb-level badness here. The error page processing will probably be dealing with persistent objects from the first transaction, carried as exception values, in tracebacks, and REQUEST attributes.

It would be a *very* bad idea to allow the second transaction to commit. Copying persistent objects across transaction in this way can lead to dangling references, and I suspect other unacceptable problems. For example, the error reporting function could store a reference to object M from the previous example. A concurrent pack could legitimately have purged it - leaving a dangling reference. I suspect sessions will be particularly vulnerable to this.

Is it bad to always do error reporting in a seperate transaction that is always aborted? I suspect so, but can find a specific example at the moment. I may comment further after sleep.

My vote goes to performing the error reporting in the original transaction. IMO this makes sense because conceptually the error report is part of that transaction. It also sidesteps all zodb problems. We definitely need to abort the transaction in the right place - *after* the error report.

________________________________________
= Comment - Entry #3 by stevea on Mar 27, 2003 1:11 pm

FYI:

Dieter's suggestion of rendering the error page in a new transaction that is either committed or aborted depending whether the error page raises an exception is pretty much the same as what happens in Zope 3.

________________________________________
= Comment - Entry #2 by efge on Mar 27, 2003 3:13 am

Here is the mentionned thread:
http://mail.zope.org/pipermail/zope-dev/2003-February/018634.html
________________________________________
= Request - Entry #1 by d.maurer on Mar 27, 2003 2:48 am

This has been discussed on zope-dev.

ZPublisher executes the error hook outside
of an explicit transaction. This means that
the state is undefined when error handling modifies
the ZODB state.

As Toby pointed out in the above mentioned thread,
the implicitly created
transaction is aborted as part of a later
request. *HOWEVER*, the association between
a transaction and a ZODB connection from the
connection pool is not fixed.
As a consequence, a different transaction can
meanwhile reuse the the affected ZODB connection.
It will see inconsistent state which has never
been committed. 

When the implicitely created
transaction is later aborted, this will lead
to asynchronous invalidation messages for the
affected cache entries. At least, if they come
too late and the new transaction has committed
meanwhile, then the uncommitted state is permanent.

To fix this, ZPublisher should perform
error handling in an explicit transaction.
Either (as Toby suggested) by delaying
the "abort" after error handling (error
handling may then see inconsistent state)
or (my suggestion) to perform it in a new
transaction which is committed, if error handling
does not raise an exception and aborted otherwise. 
==============================================================