ZODB Competing read/writes: How to find out which attribute?
Hi, I once in the while get the following warning in my Zope 3 log, which I'd like to resolve: 2009-10-07T14:35:41 WARNING ZopePublication Competing writes/reads at /BSPSite/act/++vh++http:zis.act.at:80/bankneu/++/c/acc/booklist/index.html: database conflict error (oid 0x7a5d, class BTrees.OOBTree.OOBTree, serial this txn started with 0x0381388ca6804966 2009-10-07 12:28:39.023700, serial currently committed 0x03813893b087d0cc 2009-10-07 12:35:41.374343) I wonder how I can find out, to which attribute this applies, I do know the view and the class, probably BTrees.OOBTree.OOBTree, but don't know the attribute itself. Any clues? Best Regards, Hermann -- hermann@qwer.tk GPG key ID: 299893C7 (on keyservers) FP: 0124 2584 8809 EF2A DBF9 4902 64B4 D16B 2998 93C7
Hermann Himmelbauer wrote:
Hi, I once in the while get the following warning in my Zope 3 log, which I'd like to resolve:
2009-10-07T14:35:41 WARNING ZopePublication Competing writes/reads at /BSPSite/act/++vh++http:zis.act.at:80/bankneu/++/c/acc/booklist/index.html: database conflict error (oid 0x7a5d, class BTrees.OOBTree.OOBTree, serial this txn started with 0x0381388ca6804966 2009-10-07 12:28:39.023700, serial currently committed 0x03813893b087d0cc 2009-10-07 12:35:41.374343)
That URL looks like it should only be reading the database, but the fact that you get a conflict error suggests that the code behind that URL is writing to the database. If I were you, I would first ensure that read operations never write to the database. Shane
Hi Hermann
Betreff: Re: [Zope-dev] ZODB Competing read/writes: How to find out which attribute?
Hermann Himmelbauer wrote:
Hi, I once in the while get the following warning in my Zope 3 log, which I'd like to resolve:
2009-10-07T14:35:41 WARNING ZopePublication Competing writes/reads at
/BSPSite/act/++vh++http:zis.act.at:80/bankneu/++/c/acc/booklis t/index.html:
database conflict error (oid 0x7a5d, class BTrees.OOBTree.OOBTree, serial this txn started with 0x0381388ca6804966 2009-10-07 12:28:39.023700, serial currently committed 0x03813893b087d0cc 2009-10-07 12:35:41.374343)
That URL looks like it should only be reading the database, but the fact that you get a conflict error suggests that the code behind that URL is writing to the database. If I were you, I would first ensure that read operations never write to the database.
just a hint, sometimes session get accessed with session[key] wich will force a write operation. If so, you can prevent creating a new session by using session.get(key, someDefaultValue) Regards Roger Ineichen
Shane
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Am Freitag 09 Oktober 2009 21:44:12 schrieb Roger Ineichen:
Hi Hermann
Betreff: Re: [Zope-dev] ZODB Competing read/writes: How to find out which attribute?
Hermann Himmelbauer wrote:
Hi, I once in the while get the following warning in my Zope 3
log, which
I'd like to resolve:
2009-10-07T14:35:41 WARNING ZopePublication Competing
writes/reads at
/BSPSite/act/++vh++http:zis.act.at:80/bankneu/++/c/acc/booklis
t/index.html:
database conflict error (oid 0x7a5d, class BTrees.OOBTree.OOBTree, serial this txn started with 0x0381388ca6804966 2009-10-07 12:28:39.023700, serial currently committed 0x03813893b087d0cc 2009-10-07 12:35:41.374343)
That URL looks like it should only be reading the database, but the fact that you get a conflict error suggests that the code behind that URL is writing to the database. If I were you, I would first ensure that read operations never write to the database.
just a hint, sometimes session get accessed with session[key] wich will force a write operation. If so, you can prevent creating a new session by using session.get(key, someDefaultValue)
Thanks for the info - but why is that? I always had the impression, that mydict['mykey'] does the same as mydict.get('mykey')? What exactly is written during session['mykey']? Moreover, does that concurrent read/write problem with sessions apply to one user only or to anyone who accesses the session? Best Regards, Hermann -- hermann@qwer.tk GPG key ID: 299893C7 (on keyservers) FP: 0124 2584 8809 EF2A DBF9 4902 64B4 D16B 2998 93C7
Am Freitag 09 Oktober 2009 19:12:09 schrieb Shane Hathaway:
Hermann Himmelbauer wrote:
Hi, I once in the while get the following warning in my Zope 3 log, which I'd like to resolve:
2009-10-07T14:35:41 WARNING ZopePublication Competing writes/reads at /BSPSite/act/++vh++http:zis.act.at:80/bankneu/++/c/acc/booklist/index.htm l: database conflict error (oid 0x7a5d, class BTrees.OOBTree.OOBTree, serial this txn started with 0x0381388ca6804966 2009-10-07 12:28:39.023700, serial currently committed 0x03813893b087d0cc 2009-10-07 12:35:41.374343)
That URL looks like it should only be reading the database, but the fact that you get a conflict error suggests that the code behind that URL is writing to the database. If I were you, I would first ensure that read operations never write to the database.
That's exactly the problem - it's a read operation and there should not be any write operation involved. However, it's hard to find out where the write operation in my code occurs, I can't find it in the view, maybe there's something in the authentication code, but I'd have to dig deep through my source. So It would be extremely helpful to know a faster way to find out what code triggered this warning. Best Regards, Hermann -- hermann@qwer.tk GPG key ID: 299893C7 (on keyservers) FP: 0124 2584 8809 EF2A DBF9 4902 64B4 D16B 2998 93C7
Hermann Himmelbauer wrote:
That's exactly the problem - it's a read operation and there should not be any write operation involved. However, it's hard to find out where the write operation in my code occurs, I can't find it in the view, maybe there's something in the authentication code, but I'd have to dig deep through my source. So It would be extremely helpful to know a faster way to find out what code triggered this warning.
The code that modifies something will indirectly call the register() method of the ZODB.Connection class. If you arrange for Connection.register() to raise an exception when the transaction is not supposed to modify anything, you should get a traceback that pinpoints what is doing the modification. Unfortunately, there is no API for that, so the easiest thing to do is modify the source code of the register() method in ZODB.Connection. Fortunately, this is Python, so modifying someone else's code is usually quite easy. :-) Shane
On Sat, Oct 10, 2009 at 10:00 PM, Shane Hathaway <shane@hathawaymix.org> wrote:
Hermann Himmelbauer wrote:
That's exactly the problem - it's a read operation and there should not be any write operation involved. However, it's hard to find out where the write operation in my code occurs, I can't find it in the view, maybe there's something in the authentication code, but I'd have to dig deep through my source. So It would be extremely helpful to know a faster way to find out what code triggered this warning.
The code that modifies something will indirectly call the register() method of the ZODB.Connection class. If you arrange for Connection.register() to raise an exception when the transaction is not supposed to modify anything, you should get a traceback that pinpoints what is doing the modification. Unfortunately, there is no API for that, so the easiest thing to do is modify the source code of the register() method in ZODB.Connection. Fortunately, this is Python, so modifying someone else's code is usually quite easy. :-)
Easier yet, in a development environment, add a breakpoint in register and make a read-only request to see what's causing it to be called and why. Jim -- Jim Fulton
On Sat, Oct 10, 2009 at 10:56:09AM +0200, Hermann Himmelbauer wrote:
Am Freitag 09 Oktober 2009 19:12:09 schrieb Shane Hathaway:
Hermann Himmelbauer wrote:
Hi, I once in the while get the following warning in my Zope 3 log, which I'd like to resolve:
2009-10-07T14:35:41 WARNING ZopePublication Competing writes/reads at /BSPSite/act/++vh++http:zis.act.at:80/bankneu/++/c/acc/booklist/index.htm l: database conflict error (oid 0x7a5d, class BTrees.OOBTree.OOBTree, serial this txn started with 0x0381388ca6804966 2009-10-07 12:28:39.023700, serial currently committed 0x03813893b087d0cc 2009-10-07 12:35:41.374343)
That URL looks like it should only be reading the database, but the fact that you get a conflict error suggests that the code behind that URL is writing to the database. If I were you, I would first ensure that read operations never write to the database.
That's exactly the problem - it's a read operation and there should not be any write operation involved. However, it's hard to find out where the write operation in my code occurs, I can't find it in the view, maybe there's something in the authentication code, but I'd have to dig deep through my source. So It would be extremely helpful to know a faster way to find out what code triggered this warning.
Sometimes looking at the object in question can provide some hints. You have the OID, start up zodbbrowser (http://pypi.python.org/pypi/zodbbrowser), go to @@zodbbrowser?oid=0x7a5d and see if the data sheds some link. Some things, e.g. annotations, stand out pretty clearly (and the default adapter for IAttributeAnnotatable is one of the things that has write-on-read semantics.) Sadly, the ZODB doesn't track backreferences, so there only way to find out for sure what points to your object is to do a full graph traversal. Cheers! Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development
On Sun, Oct 11, 2009 at 3:27 PM, Marius Gedminas <marius@gedmin.as> wrote: ...
Sadly, the ZODB doesn't track backreferences, so there only way to find out for sure what points to your object is to do a full graph traversal.
The zc.zodbdgc package has a database verification scripts that optionally creates a backreference database. Jim -- Jim Fulton
participants (5)
-
Hermann Himmelbauer -
Jim Fulton -
Marius Gedminas -
Roger Ineichen -
Shane Hathaway