Zope container objects can't multitask; Z2 CONFLICT mis-reported
Zope container objects can't multitask. If you try and add more than one item to a folder at a time -- even if they have different ids -- a Z2 CONFLICT error occurs. To add insult to injury, the error message sent to the user seems to be almost random. I've seen both of these: Missing doc string at: http://BEETLEJUICE:8080/folder1 The parameter, <em>id</em>, was omitted from the request. To reproduce the problem, create a folder named /folder1 on your server and run the following code (don't forget to edit the defaults!): # test Zope's ability to add more than one item to a folder simultaneously import httplib, sys, time, urllib, base64, thread, string __defaultServer = '127.0.0.1' __defaultPath = '/folder1' __defaultPort = 8080 __defaultUsername = 'zopeadmin' __defaultPassword = '<your password>' def httpPOST(dict, server, port, path, username, password): """I blame section 4.61 of the Python FAQ.""" postings = [] for key in dict.keys(): postings.append('%s=%s' % (key, urllib.quote(dict[key]))) qs = string.join(postings, '&') userpass = '%s:%s' % (username, password) httpobj = httplib.HTTP(server, port) httpobj.putrequest('POST', path) httpobj.putheader('Accept', '*/*') httpobj.putheader('Connection', 'Keep-Alive') httpobj.putheader('Content-type', 'application/x-www-form-urlencoded') httpobj.putheader('Content-length', '%d' % len(qs)) httpobj.putheader('Authorization', 'Basic %s' % base64.encodestring(userpass)) httpobj.endheaders() httpobj.send(qs) reply, msg, headers = httpobj.getreply() return reply, msg, headers, httpobj.getfile().read() def createDTMLDocument( id, title='', server=__defaultServer, port=__defaultPort, path=__defaultPath, username=__defaultUsername, password=__defaultPassword ): """Create a DTML document""" dict = { 'id': id, 'title': title } return httpPOST(dict, server, port, path + '/manage_addProduct/OFSP/addDTMLDocument', username, password) def Test(numDocs): for intId in range(numDocs): ident = thread.get_ident() id = '%d.%d' % (intId, ident) title = 'concurrency' print 'creating document %s' % id reply, msg, headers, body = createDTMLDocument(id, title) if reply != 302: # I'm expecting a redirect print 'reply status:', reply print headers print body def MultiThreadedTest(numThreads, numDocs): for threadnum in range(numThreads): thread.start_new_thread(Test, (numDocs,) ) # Test(5) MultiThreadedTest(3,3) Regards, Garth. -- <gtk@well.com>
In article <009f01bf2396$6d74c920$0b01010a@beetlejuice>, gtk <gtk@well.com> writes OK I tried to reproduce this, but have got an error on the client side. I'm running win32 on the same machine as the server and get
execfile('/python/devel/zope/import/thrash.py') creating document 0.-141813 creating document 0.-188337 creating document 0.-187821 Traceback (innermost last): File "/python/devel/zope/import/thrash.py", line 54, in Test reply, msg, headers, body = createDTMLDocument(id, title) File "/python/devel/zope/import/thras h.py", line 45, in createDTMLDocument return httpPOST(dict, server, port, path + '/manage_add...... ck.connect(host, port) File "<string>", line 1, in connect File "<string>", line 1, in connectsocket socket..errorerror: : (10061, 'winsock error')(10061, 'winsock error') -- Robin Becker
OK I tried to reproduce this, but have got an error on the client side. [...] socket..errorerror: : (10061, 'winsock error')(10061, 'winsock error')
Try switching the comment tags at the bottom: Test(5) # MultiThreadedTest(3,3) If you can run Test but not MultiThreadedTest -- don't forget to create a fresh folder1 before starting! -- then you're probably seeing the conflict in action. Regards, Garth. -- <gtk@well.com>
gtk wrote:
Zope container objects can't multitask. If you try and add more than one item to a folder at a time -- even if they have different ids -- a Z2 CONFLICT error occurs.
Adding items to a folder in separate threads does cause a conflict, since you are modifying the same object, the folder. The database detects this situation and raises a conflict error. The application detects conflict errors and retries the request up to three times. If the request doesn't succeed on the third try, then an error is returned to the user. The error should be a conflict error. Zope *is* successfully multi-tasking. The definition of successful multi-tasking is that, *if* data are written, they are written correctly. One thread doesn't overwrite partial results from another thread, causing data inconsistencies. Refusal to write data, while not desireable, is acceptable. If you have an application that has such a high write rate to a single object, such as a folder, you should consider redesigning the application to avoid this hot spot. In the future, we plan to add protocols that will allow objects to resolve conflicts at the application level. This would allow some conflicting operations, such as folder adds to be non-conflicting (assuming that the items added had different ids of course).
To add insult to injury, the error message sent to the user seems to be almost random. I've seen both of these:
Missing doc string at: http://BEETLEJUICE:8080/folder1 The parameter, <em>id</em>, was omitted from the request.
This is a bug. I have submitted it to the Collector (http://www.zope.org:8080/Collector/) on your behaf. Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
participants (3)
-
gtk -
Jim Fulton -
Robin Becker