Here I was I thought Conflict Errors only happen during writes, but I'm getting conflict errors in the strangest places: /home/zope/Zope/lib/python/Products/PageTemplates/Expressions.py line 332: o=object[name] inside our code, in a sorting function: return cmp(getattr(x, attribute), getattr(y, attribute)) Now these are "just" attribute/member accesses .. the objects holding the attributes (ie. "object", "x" and "y") are PropertyManagers and ObjectManagers. Anyone have any hints? Richard
Yeah, well... yeah. (Staring at shoes)... conflicts can also happen on.. reads. Yes. Read conflicts, they're called. This happens in the setstate method of Connection.py. There's some explanation for this that has to do with consistency of data but I don't know it; at least I dont know it well enough to explain it competently. Note that if you're patient enough, you might be able to take the consistency hit (whatever it is) and make use of the LowConflictConnection class that ships with CoreSessionTracking 0.9.. see the CST code to see how it's used. Richard Jones wrote:
Here I was I thought Conflict Errors only happen during writes, but I'm getting conflict errors in the strangest places:
/home/zope/Zope/lib/python/Products/PageTemplates/Expressions.py line 332: o=object[name]
inside our code, in a sorting function: return cmp(getattr(x, attribute), getattr(y, attribute))
Now these are "just" attribute/member accesses .. the objects holding the attributes (ie. "object", "x" and "y") are PropertyManagers and ObjectManagers.
Anyone have any hints?
Richard
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
-- Chris McDonough Zope Corporation http://www.zope.org http://www.zope.com "Killing hundreds of birds with thousands of stones"
On Thu, 01 Nov 2001 01:00:14 -0500, Chris McDonough <chrism@digicool.com> wrote:
Yeah, well... yeah. (Staring at shoes)... conflicts can also happen on.. reads. Yes. Read conflicts, they're called. This happens in the setstate method of Connection.py. There's some explanation for this that has to do with consistency of data but I don't know it; at least I dont know it well enough to explain it competently.
Zope loads objects from their persistent store into memory when they are first accessed. So if you have a transaction that touches several objects, you will be loading those objects gradually throughout the transaction. (unless they are already in memory anyway; lets assume they are not) Consider one of the last objects to be loaded..... the question is: what happens if that object has been modified in a different thread since the start of this current transaction? Ideally, Zope would load the original state of the object. That is, the state that was current at the time when this transaction started. However, this capability has not yet been developed. When it does read conflicts will not happen. Another option would be to load the new state of that object. This is what the first revision of ZODB did, and it is bad because it leads to inconsistencies. Our transaction sees some, but not all of the changes made in a different transaction. There are some objects for which this isnt a problem; I think the LowConflictConnection class that Chris referred to can allow you to exploit this. Toby Dickenson tdickenson@geminidataloggers.com
Damn, this is an excellent description. I'm gonna go add it to the developer's guide. Toby Dickenson wrote:
On Thu, 01 Nov 2001 01:00:14 -0500, Chris McDonough <chrism@digicool.com> wrote:
Yeah, well... yeah. (Staring at shoes)... conflicts can also happen on.. reads. Yes. Read conflicts, they're called. This happens in the setstate method of Connection.py. There's some explanation for this that has to do with consistency of data but I don't know it; at least I dont know it well enough to explain it competently.
Zope loads objects from their persistent store into memory when they are first accessed. So if you have a transaction that touches several objects, you will be loading those objects gradually throughout the transaction.
(unless they are already in memory anyway; lets assume they are not)
Consider one of the last objects to be loaded..... the question is: what happens if that object has been modified in a different thread since the start of this current transaction?
Ideally, Zope would load the original state of the object. That is, the state that was current at the time when this transaction started. However, this capability has not yet been developed. When it does read conflicts will not happen.
Another option would be to load the new state of that object. This is what the first revision of ZODB did, and it is bad because it leads to inconsistencies. Our transaction sees some, but not all of the changes made in a different transaction. There are some objects for which this isnt a problem; I think the LowConflictConnection class that Chris referred to can allow you to exploit this.
Toby Dickenson tdickenson@geminidataloggers.com
-- Chris McDonough Zope Corporation http://www.zope.org http://www.zope.com "Killing hundreds of birds with thousands of stones"
On Thursday 01 November 2001 17:00, Chris McDonough wrote:
Yeah, well... yeah. (Staring at shoes)... conflicts can also happen on.. reads. Yes. Read conflicts, they're called. This happens in the setstate method of Connection.py. There's some explanation for this that has to do with consistency of data but I don't know it; at least I dont know it well enough to explain it competently.
Note that if you're patient enough, you might be able to take the consistency hit (whatever it is) and make use of the LowConflictConnection class that ships with CoreSessionTracking 0.9.. see the CST code to see how it's used.
I've been looking into the standard setstate() method of Connection - and I have a couple of questions: 1. where is _invalid defined? I grepped the Zope source, and can't find it anywhere. I'm curious as to the meaning of _invalid(None). 2. _p_independent is a very interesting little method. It's only used in one place, as far as I can tell (BTrees/Length.py). If it truly did do what it says it's supposed to do - state that an object doesn't affect or rely on other objects, then you'd be able to tack a _p_independent() returning 1 onto your classes that use LowConflictConnection. I think. As it is, it seems that it's only checked for the latter statement - that the object doesn't care about the state of other objects. Hrm. I think I'll stop delving into the ZODB. I'm starting to get bad feelings. Richard
Richard Jones wrote:
On Thursday 01 November 2001 17:00, Chris McDonough wrote:
I've been looking into the standard setstate() method of Connection - and I have a couple of questions:
1. where is _invalid defined? I grepped the Zope source, and can't find it anywhere. I'm curious as to the meaning of _invalid(None).
Why of course it's the has_key method of the Connection's _invalidated attribute (see Conenction's __init__ method)! Remember, this is Jim Fulton code. ;-)
2. _p_independent is a very interesting little method. It's only used in one place, as far as I can tell (BTrees/Length.py). If it truly did do what it says it's supposed to do - state that an object doesn't affect or rely on other objects, then you'd be able to tack a _p_independent() returning 1 onto your classes that use LowConflictConnection. I think. As it is, it seems that it's only checked for the latter statement - that the object doesn't care about the state of other objects.
Sorry Richard, I don't understand that question?
Hrm. I think I'll stop delving into the ZODB. I'm starting to get bad feelings.
Creepy, huh? -- Chris McDonough Zope Corporation http://www.zope.org http://www.zope.com "Killing hundreds of birds with thousands of stones"
On Friday 02 November 2001 12:14, Chris McDonough wrote:
Richard Jones wrote:
On Thursday 01 November 2001 17:00, Chris McDonough wrote:
I've been looking into the standard setstate() method of Connection - and I have a couple of questions:
1. where is _invalid defined? I grepped the Zope source, and can't find it anywhere. I'm curious as to the meaning of _invalid(None).
Why of course it's the has_key method of the Connection's _invalidated attribute (see Conenction's __init__ method)! Remember, this is Jim Fulton code. ;-)
Ah, of course :)
2. _p_independent is a very interesting little method. It's only used in one place, as far as I can tell (BTrees/Length.py). If it truly did do what it says it's supposed to do - state that an object doesn't affect or rely on other objects, then you'd be able to tack a _p_independent() returning 1 onto your classes that use LowConflictConnection. I think. As it is, it seems that it's only checked for the latter statement - that the object doesn't care about the state of other objects.
Sorry Richard, I don't understand that question?
Nor do I. Imagine me backing away slowly :) Having said that - in our endeavour to kill off all the ConflictErrors in our code, we discovered that we had an unnecessay attribute modification deep in the code of our shop product display code. That not only caused the occasional write conflict, but also slowed our code down considerably. Removing the attribute modification has sped our product display time up by 3x (and the modification was only being done once per display.) So it hasn't all been for naught :) Richard
On Friday 02 November 2001 11:30, Richard Jones wrote:
Having said that - in our endeavour to kill off all the ConflictErrors in our code, we discovered that we had an unnecessay attribute modification deep in the code of our shop product display code. That not only caused the occasional write conflict, but also slowed our code down considerably. Removing the attribute modification has sped our product display time up by 3x (and the modification was only being done once per display.) So it hasn't all been for naught :)
Ahem. On closer inspection, this wasn't the case. We did save about 10%, but not 66% like I thought we had :) Richard
participants (3)
-
Chris McDonough -
Richard Jones -
Toby Dickenson