RE: [Zope] zope operations atomic?
Chris, Thanks for the explanation. I don't see how ApplicationLevelConflictResolution can handle the dirty read problem though. -----Original Message----- From: Chris McDonough [mailto:chrism@zope.com] Sent: Wednesday, October 31, 2001 3:11 PM To: Clark OBrien; k_vertigo@yahoo.com; zope@zope.org Subject: Re: [Zope] zope operations atomic?
Example: To count the number of files uploaded to my site I set an index as an attribute to a Folder. These are large files so they take some time to upload. What happens after two people upload at once does my index only get rolled back or do the files uploaded also get rolled back?
If the uploads finish at the same time (meaning the transactions will likely commit at the same time), a ConflictError is raised, and one of the requests is retried. (Note that the file needn't be uploaded from the remote browser again; Zope keeps the request around -- which includes the cgi data, which includes your file's data -- to retry with.) The uploader never notices any of this, except perhaps the site will become sluggish. None of this has anything to do with the counter in your example, except that a ConflictError is more likely to be raised when changing "hotspot" items like a counter. The example you provide of a counter is one that is problematic for ZODB for two reasons: 1. Multiple simultaneous commits which include counter incrementing can cause conflicts 2. The undoability of FileStorage causes the Data.fs file to grow on every successful transaction. These problems are orthogonal to each other; they are not related to each other in any way except that they both occur in the default Zope configuration. The first is solved via application level conflict resolution (http://www.zope.org/Members/jim/ZODB/ApplicationLevelConflictResoluti on) as well as a storage which supports it, the second is solved by using a nonundoing storage. So in the example you gave, you're most worried about problem #1, conflicts. To solve this particular problem, you would implement a "smart" counter that did not make itself a hotspot. There is actually already one of these that ships with Zope. It's in the BTrees.Length module, and it's a counter that does app-level conflict resolution. It's also a good module to take a look at if you're interested in seeing how conflict resolution works. Note that none of this is terribly different AFAICT than how most relational databases implement multithreadedness save for the fact that ZODB uses optimistic concurrency instead of pessimistic, allowing the app developer to be able to handle conflict scenarios in order to maximize performance. HTH, - C
What is "the dirty read problem"? ----- Original Message ----- From: "Clark OBrien" <COBrien@isis-server.vuse.vanderbilt.edu> To: "'Chris McDonough'" <chrism@zope.com>; "Clark OBrien" <COBrien@isis-server.vuse.vanderbilt.edu>; <k_vertigo@yahoo.com>; <zope@zope.org> Sent: Thursday, November 01, 2001 9:39 AM Subject: RE: [Zope] zope operations atomic?
Chris, Thanks for the explanation. I don't see how ApplicationLevelConflictResolution can handle the dirty read problem though.
-----Original Message----- From: Chris McDonough [mailto:chrism@zope.com] Sent: Wednesday, October 31, 2001 3:11 PM To: Clark OBrien; k_vertigo@yahoo.com; zope@zope.org Subject: Re: [Zope] zope operations atomic?
Example: To count the number of files uploaded to my site I set an index as an attribute to a Folder. These are large files so they take some time to upload. What happens after two people upload at once does my index only get rolled back or do the files uploaded also get rolled back?
If the uploads finish at the same time (meaning the transactions will likely commit at the same time), a ConflictError is raised, and one of the requests is retried.
(Note that the file needn't be uploaded from the remote browser again; Zope keeps the request around -- which includes the cgi data, which includes your file's data -- to retry with.)
The uploader never notices any of this, except perhaps the site will become sluggish.
None of this has anything to do with the counter in your example, except that a ConflictError is more likely to be raised when changing "hotspot" items like a counter.
The example you provide of a counter is one that is problematic for ZODB for two reasons:
1. Multiple simultaneous commits which include counter incrementing can cause conflicts
2. The undoability of FileStorage causes the Data.fs file to grow on every successful transaction.
These problems are orthogonal to each other; they are not related to each other in any way except that they both occur in the default Zope configuration. The first is solved via application level conflict resolution
(http://www.zope.org/Members/jim/ZODB/ApplicationLevelConflictResoluti
on) as well as a storage which supports it, the second is solved by using a nonundoing storage.
So in the example you gave, you're most worried about problem #1, conflicts. To solve this particular problem, you would implement a "smart" counter that did not make itself a hotspot. There is actually already one of these that ships with Zope. It's in the BTrees.Length module, and it's a counter that does app-level conflict resolution. It's also a good module to take a look at if you're interested in seeing how conflict resolution works.
Note that none of this is terribly different AFAICT than how most relational databases implement multithreadedness save for the fact that ZODB uses optimistic concurrency instead of pessimistic, allowing the app developer to be able to handle conflict scenarios in order to maximize performance.
HTH,
- C
participants (2)
-
Chris McDonough -
Clark OBrien