Hi, Whenever there's a lot of writes and the occasional read going on, we get the following error: 2000-11-19T16:03:44 INFO(0) Z2 CONFLICT Competing writes at, /aFolder/anObject Traceback (innermost last): File D:\Program Files\Zope\lib\python\ZPublisher\Publish.py, line 175, in publ ish File D:\Program Files\Zope\lib\python\Zope\__init__.py, line 235, in commit File D:\Program Files\Zope\lib\python\ZODB\Transaction.py, line 251, in commit File D:\Program Files\Zope\lib\python\ZODB\Connection.py, line 267, in commit ConflictError: y Can anyone shed any light on: -what this means -how bad it is -how we can make it stop happening cheers, Chris
It happens when more than one thread tries to modify the same object in storage at the same time. It's Zope's equivalent of record-locking... It's normal. There's no corruption or anything, it's telling you that it avoided a write that might have caused problems. Zope tries three times to retry the write. If the object is still locked by another database connection after three writes, it propogates the exception up to the app level. That's the error you see. Careful application coding can reduce the chance of conflict errors. I've seen conflict errors pop up a lot when the catalog is used for whatever reason, especially with catalogaware. I bet it happens on a catalog write for you. I think this may have something to do with the current BTree implementation's propensity to write child-parent relationships all the way up and down the tree every time a node changes. I know your next questions are going to be "why?" and "how do I fix it?" and "why don't you fix it?". Chris Petrilli did some work on the catalog for 2.2.3 that might have a positive impact. The catalog is (I think) being actively worked on.... if you can nail it down some, it'd probably be helpful to figure out when the conflicterror occurs and pass it along to Chris in the collector. Besides that, I don't have any specific hints for you, besides waiting for the new BTree implementation. The answer to "when?": I have no idea. HTH, C ----- Original Message ----- From: "Chris Withers" <chrisw@nipltd.com> To: <zope-dev@zope.org> Sent: Monday, November 20, 2000 10:46 AM Subject: [Zope-dev] Conflict Errors
Hi,
Whenever there's a lot of writes and the occasional read going on, we get the following error:
2000-11-19T16:03:44 INFO(0) Z2 CONFLICT Competing writes at, /aFolder/anObject Traceback (innermost last): File D:\Program Files\Zope\lib\python\ZPublisher\Publish.py, line 175, in publ ish File D:\Program Files\Zope\lib\python\Zope\__init__.py, line 235, in commit File D:\Program Files\Zope\lib\python\ZODB\Transaction.py, line 251, in commit
File D:\Program Files\Zope\lib\python\ZODB\Connection.py, line 267, in commit ConflictError: y
Can anyone shed any light on: -what this means -how bad it is -how we can make it stop happening
cheers,
Chris
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Thanks for the reply :-) Chris McDonough wrote:
It happens when more than one thread tries to modify the same object in storage at the same time. It's Zope's equivalent of record-locking... It's normal. There's no corruption or anything, it's telling you that it avoided a write that might have caused problems. Zope tries three times to retry the write. If the object is still locked by another database connection after three writes, it propogates the exception up to the app level. That's the error you see.
Hurm, well, it appears to happen when we're doing a lot of sequential write to a product (automated bulk data upload), but the thing that triggers it off is actually trying to read a page from the same area of the site... confusing. Although that could also be a one-off coincidence... You say no corruption or anything, but if a submit results in that error, does the submitted form data get processed or not?
Careful application coding can reduce the chance of conflict errors.
Can you describe what you mean by careful application coding?
I bet it happens on a catalog write for you.
No, for once it has nothing to do with the catalog. ;-) This is a python product of ours that stores lots of data in somewhat nested, persistent dictionaries. How can this be improved so this doesn't happen?
I know your next questions are going to be "why?" and "how do I fix it?" and "why don't you fix it?".
While I'd love it if you'd fix our code for free, it's probably beyond the OSS remit ;-) Any clues about how to make our code 'less bad' would be cool... cheers, Chris
Chris Withers wrote:
i'm no expert but deeply nested data structures in a single object are rife with possibilities for conflict errors. if data might be changing concurrently in the structures or frequently changing you would be better off to switch the structure to nested persistent objects.
No, for once it has nothing to do with the catalog. ;-) This is a python product of ours that stores lots of data in somewhat nested, persistent dictionaries. How can this be improved so this doesn't happen?
I know your next questions are going to be "why?" and "how do I fix it?" and "why don't you fix it?".
While I'd love it if you'd fix our code for free, it's probably beyond the OSS remit ;-) Any clues about how to make our code 'less bad' would be cool...
cheers,
Chris
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Hurm, well, it appears to happen when we're doing a lot of sequential write to a product (automated bulk data upload), but the thing that triggers it off is actually trying to read a page from the same area of the site... confusing. Although that could also be a one-off coincidence...
Conflict errors can be raised by three things: the storage, the transaction, and the connection. The storage will only raise the conflict error on a write, this is for sure. The transaction and the connection objects I looked at a little today, and it's unclear .. although as I understand it, the idea of a ConflictError is to prevent the modification of an object by two threads at the same time, and should have nothing to do with reads.
You say no corruption or anything, but if a submit results in that error, does the submitted form data get processed or not?
No... the transaction is aborted...
Careful application coding can reduce the chance of conflict errors.
Can you describe what you mean by careful application coding?
Ah I knew I wasn't gonna get away with that one. ;-) I probably shoulda kept my mouth shut. If you're writing a ZODB-based app that uses multiple database connections (like Zope ;-), you should take care to stagger high-load writes against an object in the threads that use those database connections. The ZODB tries hard to avoid error conditions for you by retrying conflicting writes, but if you really pound the snot out of a particular object, it'll let you know. That said, I doubt I can be more specific, because to be honest I don't know the *real* answer myself. Sigh. I think I need to learn more about pickling (to figure out how the objects get split up into component pieces) and step through a real-world ConflictError situation. Neither of which I have time to do right now... sorry, Chris.
I bet it happens on a catalog write for you.
No, for once it has nothing to do with the catalog. ;-) This is a python product of ours that stores lots of data in somewhat nested, persistent dictionaries. How can this be improved so this doesn't happen?
Can you replicate the error on demand? Or is it intermittent? This should be something covered in developer docs (along with a zillion other things). I suppose a gross hack would be to raise the number of write retries on conflicts in the ZODB code if you didn't want to analyze your app code. But y'know, as I think more about it, I'm afraid I'm gonna need to bail on this one by saying "I don't know" because it's one of those things where learning exactly what is going on can chew up time like nobody's business, because I'd need to step through the code line-by-line. I shouldn't even be in here. :-( Sorry, C
Chris McDonough wrote:
The ZODB tries hard to avoid error conditions for you by retrying conflicting writes, but if you really pound the snot out of a particular object, it'll let you know.
That's gotta be one for the Zope quotes board, if I could onyl remember the URL :-)
and step through a real-world ConflictError situation. Neither of which I have time to do right now... sorry, Chris.
this one by saying "I don't know" because it's one of those things where learning exactly what is going on can chew up time like nobody's business, because I'd need to step through the code line-by-line. I shouldn't even be in here. :-(
Hey, no problem, the help was much appreciated :-) However, if anyone does know what constitutes a 'seperate object' in ZODB terms, it be really useful to know. I'm guessing a class which inherits from Persistent? Dictionaries don't, 'cos they're python builtins, which I'm pretty certain was the problem in my case... cheers, Chris
Y'know since replying to this message originally, I've learned a lot about pickled representations of Zope objects and I think I can answer this more cogently: - Instances of classes which inherit from Persistence.Persistent are represented as an individual pickle. Their attributes are contained in this pickle, unless those attributes are themselves instances of something that inherits from Persistence.Persistent. - References to persistent objects in other objects are kept as an object identifier. This means, for example, that a dictionary of integers to persistent object instances looks something like this in a pickle:: {1:<<oid 1>>, 2:<<oid 2>>, ...} If this dictionary is an attribute of a persistent object, adding, changing, or removing items from it will modify its containing persistent object. It was this "reference by oid" concept that turned the light on for me here, hopefully it can do the same for you. One of the potential big wins with the current Zope implementation of a BTree is that it exposes a mapping interface, so you can treat it just like a dictionary. However, internally, it's composed of a number of separate persistent objects that can be changed independently of each other. This means that changing one item in a BTree will change one persistent object that makes up a BTree, but will not necessarily effect the other persistent objects that compose it. Unortunately, currently, the BTree implementation has some problems that cause that *not* to be the case in many situations, and independent writes to it often conflict. We are working on another BTree implementation to resolve the issue. ----- Original Message ----- From: "Chris Withers" <chrisw@nipltd.com> To: "Chris McDonough" <chrism@digicool.com> Cc: <zope-dev@zope.org> Sent: Tuesday, January 02, 2001 11:50 AM Subject: Re: [Zope-dev] Conflict Errors
Chris McDonough wrote:
The ZODB tries hard to avoid error conditions for you by retrying conflicting writes,
but
if you really pound the snot out of a particular object, it'll let you know.
That's gotta be one for the Zope quotes board, if I could onyl remember the URL :-)
and step through a real-world ConflictError situation. Neither of which I have time to do right now... sorry, Chris.
this one by saying "I don't know" because it's one of those things where learning exactly what is going on can chew up time like nobody's business, because I'd need to step through the code line-by-line. I shouldn't even be in here. :-(
Hey, no problem, the help was much appreciated :-)
However, if anyone does know what constitutes a 'seperate object' in ZODB terms, it be really useful to know. I'm guessing a class which inherits from Persistent? Dictionaries don't, 'cos they're python builtins, which I'm pretty certain was the problem in my case...
cheers,
Chris
On Tue, 02 Jan 2001 16:50:49 +0000, Chris Withers <chrisw@nipltd.com> wrote:
However, if anyone does know what constitutes a 'seperate object' in ZODB terms, it be really useful to know. I'm guessing a class which inherits from Persistent? Dictionaries don't, 'cos they're python builtins, which I'm pretty certain was the problem in my case...
That is exactly correct. Toby Dickenson tdickenson@geminidataloggers.com
On Tue, 21 Nov 2000 17:03:17 +0000, Chris Withers <chrisw@nipltd.com> wrote:
It happens when more than one thread tries to modify the same object in storage at the same time. It's Zope's equivalent of record-locking... It's normal. There's no corruption or anything, it's telling you that it avoided a write that might have caused problems. Zope tries three times to retry the write. If the object is still locked by another database connection after three writes, it propogates the exception up to the app level. That's the error you see.
Hurm, well, it appears to happen when we're doing a lot of sequential write to a product (automated bulk data upload), but the thing that triggers it off is actually trying to read a page from the same area of the site... confusing. Although that could also be a one-off coincidence...
ZODB marks each persistent object that is modified during a transaction. Before committing, it checks whether any of those objects have been modified-and-commited in another transaction. If it has, it raises a ConflictError. If your 'page read' really doesnt modify any objects (check your undo log, or use tranalyzer) then it cant cause a ConflictError.
You say no corruption or anything, but if a submit results in that error, does the submitted form data get processed or not?
As Chris explianed, the publisher will retry up to three ConflictErrors. If you see this message only in a log then it probably suceeded on one of those retries. If you see it in a traceback returned over http then the transaction has been aborted. No corruption, however your request has not been processed.
Careful application coding can reduce the chance of conflict errors.
Can you describe what you mean by careful application coding?
"Minimise the chance of a single persistent object being modified by two concurrent transactions." Some examples: 1. The much-maligned ZODB-page-hit-counter is bad in this respect. Each 'read' of a page modifies the counter object, causing the second concurrent read (which is actually a write) to raise ConflictError. 2. Folder objects are pretty good, although you cant concurrently add two objects to a folder since both transactions modify the folder object. 3. If you need something folderlike with a higher hit rate, it is possible to avoid some conflicts by splitting the folder. For example, having one sub-folder for each initial letter of the id. You can add objects 'chrisw' and 'tdickenson' concurrently (into subfolders with the id 'c' and 't'), however concurrently adding 'chrisw' and 'chrism' would cause one ConflictErrorm because they both modify the sub-folder 'c'. (BTreeFolders have the same advantage, for the same reason, only neater) Toby Dickenson tdickenson@geminidataloggers.com
participants (4)
-
Chris McDonough -
Chris Withers -
Ender -
Toby Dickenson