[Zope-dev] zope.sqlalchemy locks up transaction

[Hermann Himmelbauer]

Am Freitag 19 September 2008 11:59:55 schrieb Laurence Rowe:
> 2008/9/19 Hermann Himmelbauer <dusty at qwer.tk>:
> > Am Donnerstag 18 September 2008 07:08:56 schrieb Dieter Maurer:
> >> Brian Sutherland wrote at 2008-9-17 12:33 +0200:
> >> >On Tue, Sep 16, 2008 at 02:01:07PM +0100, Laurence Rowe wrote:
> >> >> Brian Sutherland wrote:
> >> >> > Hi,
> >> >> >
> >> >> > I've recently seen a situation where zope.sqlalchemy locked up the
> >> >> > transaction machinery. I'm not sure exactly what happened, but have
> >> >> > attached a failing test for at least one bug which may have caused
> >> >> > it. Hopefully it's self explanatory;)
> >> >> >
> >> >> > If someone could help me solve this, that would be great!
> >> >>
> >> >> Could you try this with latest trunk. I checked in a fix the other
> >> >> day that may help.
> >> >
> >> >I just checked in a fix, please feel free to comment on/revert it if
> >> >it's not up to standard:)
> >>
> >> It looks not yet right to clear the state in "tpc_vote" when
> >> a two phase commit is used (which is now supported by "SQLAlchemy").
> >>
> >> In addition, there may be a problem in case "session.close()" raises
> >> an exception. Then, "_finish" would not be called.
> >
> > This may not be connected, but I just wanted to point out a problem with
> > configurations with "autoflush=False". If I change some database object
> > attributes in the session but do not call session.flush(), the data is
> > not stored into the database, as no commit() is called, which would then
> > flush out/commit the data.
> >
> > So perhaps this is something to think about, or, at least something to be
> > denoted in the documentation.
> >
> > I circumvented the problem by setting autoflush=True and modifying my
> > code so that it works (although it's quite complicated).
>
> I don't think this is related to the above problem, but it may be a
> bug in SQLAlchemy 0.4 whose behaviour I have replicated.
>
> In 0.4 orm.session during a commit flush is called:
>
> 245 	        if self.autoflush:
> 246 	            self.session.flush()
>
> whereas in 0.5 flush is called unconditionally so long as there is not
> a flush in progress.
>
> 358 	        if not self.session._flushing:
> 359 	            self.session.flush()
>
> The documentation for 0.4 appears to be misleading:
>
> commit() serves two purposes; it issues a flush() unconditionally to
> persist any remaining pending changes.

Ah, I see, that's really misleading.

> I think I should change zope.sqlalchemy to flush during it's tpc_begin
> phase. This should only be done if a flush is not already in progress,
> as otherwise sqlalchemy would raise an error. 

That's maybe a good idea.

> self.session._flushing 
> is only in 0.5 though, so maybe it is time to up the dependency. 

What do you mean with "up the dependency"? If you mean dropping support for SA 
0.4, please, *please* don't do that (0.5's autoflushing breaks things here 
and Database support for MaxDB/SAPDB is somehow flawed.). Due to many 
incompatibilites, I cannot switch to SA 0.5, so I'd really appreciate support 
for SA 0.4!

> Or 
> maybe it is impossible for it to be in the flushing state at that
> point anyway, as all user code will have completed.

If I understand it right, the problem does not occur in SA 0.5, right? If 
that's the case, maybe it is enough to outline the behaviour of SA 0.4 in the 
documentation and leave it as it is.

Best Regards,
Hermann

-- 
hermann at qwer.tk
GPG key ID: 299893C7 (on keyservers)
FP: 0124 2584 8809 EF2A DBF9  4902 64B4 D16B 2998 93C7