[Zope] Zope2: Handling multiple object creations at once from multiple clients and processes?

[Sebastian Tänzer]

Hello ZOPE users,

I am dealing with the famous (Write) ConflictErrors in a module that is fetching remote files from a remote URL with the requests lib, storing them as objects with the binary data residing on the file system (no blob is stored in the ZODB).

The upload/import routine works by using jQuery’s ajax() and POSTing to the upload handler and using SSE to return a progress while the import is running and before creating the actual object inside the ZODB.

The first process of course always works fine, but when a second or third starts these trigger a ConflictError and are beeing restarted, i.e. all selected remote files are downloaded again and then create the objects inside the ZODB. All objects are created inside the same tree structure and most of the time inside the same Folder.

The conflict’s object oid mentioned in the error message actually is the new object beeing created. I first thought this had sth. to do with writing to the same object while the addObject routine is running, but it does not seem so. What I don’t understand is why the first process writes to something that triggers the conflict with the second process.

What’s the recommended way of dealing with such mass creation of objects running at the same time that are triggered by AJAX calls on the same context without creating conflicts? 

Best
Sebastian