September 2008 - Zope-Dev

Re: [Zope-dev] zope.sqlalchemy locks up transaction
by Hermann Himmelbauer 19 Sep '08

19 Sep '08

Am Freitag 19 September 2008 11:59:55 schrieb Laurence Rowe: > 2008/9/19 Hermann Himmelbauer <dusty(a)qwer.tk>: > > Am Donnerstag 18 September 2008 07:08:56 schrieb Dieter Maurer: > >> Brian Sutherland wrote at 2008-9-17 12:33 +0200: > >> >On Tue, Sep 16, 2008 at 02:01:07PM +0100, Laurence Rowe wrote: > >> >> Brian Sutherland wrote: > >> >> > Hi, > >> >> > > >> >> > I've recently seen a situation where zope.sqlalchemy locked up the > >> >> > transaction machinery. I'm not sure exactly what happened, but have > >> >> > attached a failing test for at least one bug which may have caused > >> >> > it. Hopefully it's self explanatory;) > >> >> > > >> >> > If someone could help me solve this, that would be great! > >> >> > >> >> Could you try this with latest trunk. I checked in a fix the other > >> >> day that may help. > >> > > >> >I just checked in a fix, please feel free to comment on/revert it if > >> >it's not up to standard:) > >> > >> It looks not yet right to clear the state in "tpc_vote" when > >> a two phase commit is used (which is now supported by "SQLAlchemy"). > >> > >> In addition, there may be a problem in case "session.close()" raises > >> an exception. Then, "_finish" would not be called. > > > > This may not be connected, but I just wanted to point out a problem with > > configurations with "autoflush=False". If I change some database object > > attributes in the session but do not call session.flush(), the data is > > not stored into the database, as no commit() is called, which would then > > flush out/commit the data. > > > > So perhaps this is something to think about, or, at least something to be > > denoted in the documentation. > > > > I circumvented the problem by setting autoflush=True and modifying my > > code so that it works (although it's quite complicated). > > I don't think this is related to the above problem, but it may be a > bug in SQLAlchemy 0.4 whose behaviour I have replicated. > > In 0.4 orm.session during a commit flush is called: > > 245 if self.autoflush: > 246 self.session.flush() > > whereas in 0.5 flush is called unconditionally so long as there is not > a flush in progress. > > 358 if not self.session._flushing: > 359 self.session.flush() > > The documentation for 0.4 appears to be misleading: > > commit() serves two purposes; it issues a flush() unconditionally to > persist any remaining pending changes. Ah, I see, that's really misleading. > I think I should change zope.sqlalchemy to flush during it's tpc_begin > phase. This should only be done if a flush is not already in progress, > as otherwise sqlalchemy would raise an error. That's maybe a good idea. > self.session._flushing > is only in 0.5 though, so maybe it is time to up the dependency. What do you mean with "up the dependency"? If you mean dropping support for SA 0.4, please, *please* don't do that (0.5's autoflushing breaks things here and Database support for MaxDB/SAPDB is somehow flawed.). Due to many incompatibilites, I cannot switch to SA 0.5, so I'd really appreciate support for SA 0.4! > Or > maybe it is impossible for it to be in the flushing state at that > point anyway, as all user code will have completed. If I understand it right, the problem does not occur in SA 0.5, right? If that's the case, maybe it is enough to outline the behaviour of SA 0.4 in the documentation and leave it as it is. Best Regards, Hermann -- hermann(a)qwer.tk GPG key ID: 299893C7 (on keyservers) FP: 0124 2584 8809 EF2A DBF9 4902 64B4 D16B 2998 93C7

1 0

Zope Tests: 2 OK, 3 Failed
by Zope Tests Summarizer 19 Sep '08

19 Sep '08

Summary of messages to the zope-tests list. Period Thu Sep 18 11:00:00 2008 UTC to Fri Sep 19 11:00:00 2008 UTC. There were 5 messages: 5 from Zope Tests. Test failures ------------- Subject: FAILED (failures=1) : Zope-2.10 Python-2.4.4 : Linux From: Zope Tests Date: Thu Sep 18 20:54:53 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010187.html Subject: FAILED (failures=1) : Zope-2.11 Python-2.4.4 : Linux From: Zope Tests Date: Thu Sep 18 20:56:23 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010188.html Subject: FAILED (failures=1) : Zope-trunk Python-2.4.4 : Linux From: Zope Tests Date: Thu Sep 18 20:57:53 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010189.html Tests passed OK --------------- Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Tests Date: Thu Sep 18 20:51:52 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010185.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Tests Date: Thu Sep 18 20:53:22 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010186.html

1 0

Re: [Zope-dev] zope.sqlalchemy locks up transaction
by Laurence Rowe 19 Sep '08

19 Sep '08

2008/9/19 Hermann Himmelbauer <dusty(a)qwer.tk>: > Am Donnerstag 18 September 2008 07:08:56 schrieb Dieter Maurer: >> Brian Sutherland wrote at 2008-9-17 12:33 +0200: >> >On Tue, Sep 16, 2008 at 02:01:07PM +0100, Laurence Rowe wrote: >> >> Brian Sutherland wrote: >> >> > Hi, >> >> > >> >> > I've recently seen a situation where zope.sqlalchemy locked up the >> >> > transaction machinery. I'm not sure exactly what happened, but have >> >> > attached a failing test for at least one bug which may have caused it. >> >> > Hopefully it's self explanatory;) >> >> > >> >> > If someone could help me solve this, that would be great! >> >> >> >> Could you try this with latest trunk. I checked in a fix the other day >> >> that may help. >> > >> >I just checked in a fix, please feel free to comment on/revert it if >> >it's not up to standard:) >> >> It looks not yet right to clear the state in "tpc_vote" when >> a two phase commit is used (which is now supported by "SQLAlchemy"). >> >> In addition, there may be a problem in case "session.close()" raises >> an exception. Then, "_finish" would not be called. > > This may not be connected, but I just wanted to point out a problem with > configurations with "autoflush=False". If I change some database object > attributes in the session but do not call session.flush(), the data is not > stored into the database, as no commit() is called, which would then flush > out/commit the data. > > So perhaps this is something to think about, or, at least something to be > denoted in the documentation. > > I circumvented the problem by setting autoflush=True and modifying my code so > that it works (although it's quite complicated). I don't think this is related to the above problem, but it may be a bug in SQLAlchemy 0.4 whose behaviour I have replicated. In 0.4 orm.session during a commit flush is called: 245 if self.autoflush: 246 self.session.flush() whereas in 0.5 flush is called unconditionally so long as there is not a flush in progress. 358 if not self.session._flushing: 359 self.session.flush() The documentation for 0.4 appears to be misleading: commit() serves two purposes; it issues a flush() unconditionally to persist any remaining pending changes. I think I should change zope.sqlalchemy to flush during it's tpc_begin phase. This should only be done if a flush is not already in progress, as otherwise sqlalchemy would raise an error. self.session._flushing is only in 0.5 though, so maybe it is time to up the dependency. Or maybe it is impossible for it to be in the flushing state at that point anyway, as all user code will have completed. Laurence

1 0

zope.sqlalchemy locks up transaction
by Brian Sutherland 19 Sep '08

19 Sep '08

Hi, I've recently seen a situation where zope.sqlalchemy locked up the transaction machinery. I'm not sure exactly what happened, but have attached a failing test for at least one bug which may have caused it. Hopefully it's self explanatory;) If someone could help me solve this, that would be great! -- Brian Sutherland

4 10

z3c.dav issues
by Jayarajan Jn 19 Sep '08

19 Sep '08

Hi friends, i am doing some prototypes to try webDAV with zope3. I am using z3c.extfile to store files. I tried the zope.app.dav that is coming along with z3 distribution. it works fine for zope.app.file and zope.app.folder. it partialy works with z3c.extfile too. ie, i can download extfiles Anyways i saw in http://pypi.python.org/pypi/zope.app.dav/3.4.1 that z3c.dav is advanced than zope.app.dav. So i tries z3c.dav also. But i got error while accessing the / folder through webDAV. I have been using debian's(Gnome) local filesystem browser to do this but i get the following error on my weDAV client """ cannot open '/' The filename "/" indicates that this file is of type "x-directory/webdav". The contents of the file indicate that the file is of type "HTML document". If you open this file, the file might present a security risk to your system. Do not open the file unless you created the file yourself, or received the file from a trusted source. To open the file, rename the file to the correct extension for "HTML document", then open the file normally. Alternatively, use the Open With menu to choose a specific application for the file. """ May be i am missing something :? This is how i proceeded 1) started the zopeserver 2) connected to it via webDAV with server:localhost, port:8080, user:admin. 3) it works fine and displays the content of root folder. I can view,add, delete download etc..But its done by zope.app.dav now same thing with z3c.dav 1) (easy_install -N)ed z3c.dav, z3c.etree, z3c.conditionalviews 2) included the z3c.dav package and disabled zope.app.dav . (inside package-include) 3) try to access server through webDAV (exactly same as before) now it fires above said error. can any one tell me where i am missing? i have seen the z3c.davapp.* packages. Is there any work going to support extfile like this? i have adapted the z3c.davapp.zopeappfile codes to write a similar package for z3c.extfile too. But i cant test it properly if z3c.dav itself is not working for me :/ Note: i am using zope3.3.1+py2.4 setup And like many others demand i also would like to see a HOWTO for z3c.dav. Thanks to all in advance. jayaraj.

1 0

Zope Tests: 2 OK, 3 Failed
by Zope Tests Summarizer 18 Sep '08

18 Sep '08

Summary of messages to the zope-tests list. Period Wed Sep 17 11:00:00 2008 UTC to Thu Sep 18 11:00:00 2008 UTC. There were 5 messages: 5 from Zope Tests. Test failures ------------- Subject: FAILED (failures=1) : Zope-2.10 Python-2.4.4 : Linux From: Zope Tests Date: Wed Sep 17 20:54:36 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010182.html Subject: FAILED (failures=1) : Zope-2.11 Python-2.4.4 : Linux From: Zope Tests Date: Wed Sep 17 20:56:06 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010183.html Subject: FAILED (failures=1) : Zope-trunk Python-2.4.4 : Linux From: Zope Tests Date: Wed Sep 17 20:57:36 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010184.html Tests passed OK --------------- Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Tests Date: Wed Sep 17 20:51:35 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010180.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Tests Date: Wed Sep 17 20:53:06 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010181.html

1 0

Zope 2: Unicode object ids and PropertyManager properties
by Paul Winkler 17 Sep '08

17 Sep '08

Hi, Is there a concensus on the best way in zope 2 to handle non-ascii object IDs? The current restrictions are based on a very old, gradually updated regex which still isn't right, see for example https://bugs.launchpad.net/zope2/+bug/143616 And there is a lot of code sprinkled around which assumes ascii, strings, for example in OFS.ObjectManager and OFS.CopySupport there are a couple like this: def manage_cutObjects(self, ids=None, REQUEST=None): ... if type(ids) is type(''): ids=[ids] I was about to start changing such lines to use isinstance(ids, basestring) but I thought better of it. URLs need to be hex-escaped into ASCII anyway (and there's no standard I'm aware of regarding what encoding to use before escaping; for example, chinese wikipedia appears to encode their URLs as iso8859-15 before escaping, for some bizarre reason. Either that, or some encoding I can't find which has a lot of overlap with it). So maybe we should store IDs hex-escaped; in that case, ascii is a feature, not a bug. Even if that's wrong, the ascii assumption seems to be so widespread in Zope 2 that I think, short of a full audit and a comprehensive plan, gradually using isinstance(foo, basestring) might just be false advertising that leads people into trouble. Thoughts? While looking at this, I also happened to look at OFS.PropertyManager, which doesn't explicitly handle unicode for 'string' properties - it calls ZPublisher.Converters.field2string, which encodes unicode strings into default_encoding *on the way in*. That seems bad to me: if the default encoding configuration ever changes, you'd have to write a potentially huge migration to avoid being left with stored properties in a horrid mixture of encodings. Are we stuck with that forever? Why can't we store unicode string properties as native python unicode and encode on the way out? -- Paul Winkler http://www.slinkp.com

2 1

Zope Tests: 2 OK, 3 Failed
by Zope Tests Summarizer 17 Sep '08

17 Sep '08

Summary of messages to the zope-tests list. Period Tue Sep 16 11:00:00 2008 UTC to Wed Sep 17 11:00:00 2008 UTC. There were 5 messages: 5 from Zope Tests. Test failures ------------- Subject: FAILED (failures=1) : Zope-2.10 Python-2.4.4 : Linux From: Zope Tests Date: Tue Sep 16 20:54:07 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010177.html Subject: FAILED (failures=1) : Zope-2.11 Python-2.4.4 : Linux From: Zope Tests Date: Tue Sep 16 20:55:37 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010178.html Subject: FAILED (failures=1) : Zope-trunk Python-2.4.4 : Linux From: Zope Tests Date: Tue Sep 16 20:57:07 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010179.html Tests passed OK --------------- Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Tests Date: Tue Sep 16 20:51:07 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010175.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Tests Date: Tue Sep 16 20:52:37 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-September/010176.html

1 0

lxml and z3c.form strategy?
by Roger Ineichen 16 Sep '08

16 Sep '08

Malthe, Stephan I decided after along night, trying to compile lxml on a windows box, that we will not use lxml and lxml based packages like z3c.form, z3c.template, z3c.macro etc at some of our windows servers. We have to find a strategy to keep this packages free of lxml and implement the z3c.pt support in this packages in a different way. I really like your lxml based z3c.pt work. It's nice and incredible fast. What do you think about that? I know it's a hugh amount of work, but I'm willing to help with the refactoring if you like. Regards Roger Ineichen _____________________________ END OF MESSAGE

5 5

Re: [Zope-dev] z3c.form and lxml
by Martijn Faassen 16 Sep '08

16 Sep '08

Hey, On Mon, Sep 15, 2008 at 4:47 PM, Roger Ineichen <admin(a)projekt01.ch> wrote: [snip] > Martijn, > do you you see any chance to keep the windows egg in > sync with the linux distribution at any time if w new > release comes out? As far as I'm aware Sidnei has been taking care of this for quite a while. Regards, Martijn

3 2