RE: [Zope] zope operations atomic?
Example: To count the number of files uploaded to my site I set an index as an attribute to a Folder. These are large files so they take some time to upload. What happens after two people upload at once does my index only get rolled back or do the files uploaded also get rolled back? This is what I meant anyway - I will read the ZODB3 that Dieter mentioned -----Original Message----- From: Chris McDonough [mailto:chrism@zope.com] Sent: Wednesday, October 31, 2001 1:44 PM To: Clark OBrien; k_vertigo@yahoo.com; zope@zope.org Subject: Re: [Zope] zope operations atomic?
I don't want to be dense but it sounds like for each thread a deep copy of the items it accesses- Folder for example- is made. When session ends or something triggers a commit a check is made for conflicts.
A commit is triggered by the finalization of every HTTP request.
This sounds a bit scarry because if you have a script changing resources like a Folders it is almost certain then that there will be a conflict. At this point a retry still find conflicts. I would rather have heard the Folder was a critical section or methods like manage_changeProperties were atomic. IMHOP it should be so.
Though it sounds like you have a valid concern, I can't seem to pick it out of that series of words. ;-) What is it that you're worried about? - C
Example: To count the number of files uploaded to my site I set an index as an attribute to a Folder. These are large files so they take some time to upload. What happens after two people upload at once does my index only get rolled back or do the files uploaded also get rolled back?
If the uploads finish at the same time (meaning the transactions will likely commit at the same time), a ConflictError is raised, and one of the requests is retried. (Note that the file needn't be uploaded from the remote browser again; Zope keeps the request around -- which includes the cgi data, which includes your file's data -- to retry with.) The uploader never notices any of this, except perhaps the site will become sluggish. None of this has anything to do with the counter in your example, except that a ConflictError is more likely to be raised when changing "hotspot" items like a counter. The example you provide of a counter is one that is problematic for ZODB for two reasons: 1. Multiple simultaneous commits which include counter incrementing can cause conflicts 2. The undoability of FileStorage causes the Data.fs file to grow on every successful transaction. These problems are orthogonal to each other; they are not related to each other in any way except that they both occur in the default Zope configuration. The first is solved via application level conflict resolution (http://www.zope.org/Members/jim/ZODB/ApplicationLevelConflictResoluti on) as well as a storage which supports it, the second is solved by using a nonundoing storage. So in the example you gave, you're most worried about problem #1, conflicts. To solve this particular problem, you would implement a "smart" counter that did not make itself a hotspot. There is actually already one of these that ships with Zope. It's in the BTrees.Length module, and it's a counter that does app-level conflict resolution. It's also a good module to take a look at if you're interested in seeing how conflict resolution works. Note that none of this is terribly different AFAICT than how most relational databases implement multithreadedness save for the fact that ZODB uses optimistic concurrency instead of pessimistic, allowing the app developer to be able to handle conflict scenarios in order to maximize performance. HTH, - C
participants (2)
-
Chris McDonough -
Clark OBrien