I've created an object to record some information (datetime, url, username) for each file downloaded from the site for later searching (CMF 1.3.1, Zope 2.6.1). To do this, I have a BTreeFolder with a private ZCatalog and put an new instance of a DownloadRecord object (my own object that just has string members holding the fields to be queried later) into the folder when the file is requested. I'm running into conflict errors; I think because the download record object is added to a single folder so concurrent downloads on different threads are competing trying to modified the folder. Because the downloads will take a while (they're 1-5 MB), the transaction is not committed until the download is complete. With a large download it seems that window for conflicts is large. I've been googling and reading about conflict errors but haven't found a solution yet. I've tried committing the a sub-transaction but that didn't help. I'm thinking of committing the default transaction before sending the file, but I'm not sure of the implications of doing that. Any ideas? Thanks, Jeff
On Tuesday 07 October 2003 19:10, Jeff Youel wrote:
I'm running into conflict errors; I think because the download record object is added to a single folder so concurrent downloads on different threads are competing trying to modified the folder.
That makes sense. One solution is to replace that folder object with something folder-like that implements "application level conflict resolution". That will allow your new folder-like object to 'merge' the additions made by two concurrent transactions. Why not add that to an ordinary folder? You could, but Folders have a wide interface and maintain alot of other state. It will be easier to merge a simpler object.
I've been googling and reading about conflict errors but haven't found a solution yet. I've tried committing the a sub-transaction but that didn't help.
Thats right. Write conflicts are checked by the storage, at the end of the full transaction. I hope this helps -- Toby Dickenson
Toby Dickenson wrote:
Thats right. Write conflicts are checked by the storage, at the end of the full transaction.
I hope this helps
Thanks. I'm hoping to avoid using the conflict resolution by avoiding the conflicts as much as possible. I tried committing the full transaction and beginning a new one during the request processing, but that didn't seem to help. Is it reasonable to start a new thread with a new db connection and use it to commit the changes so the database write is out-of-band wrt the request's transaction? Jeff
Jeff Youel wrote at 2003-10-9 09:15 -0700:
... Is it reasonable to start a new thread with a new db connection and use it to commit the changes so the database write is out-of-band wrt the request's transaction?
"CatalogQueue" does something like this. But beware, it is *very* difficult to get this right. It is usually desastrous to pass persistent objects between threads. Especially, you cannot update a persistent object in one thread and then pass it on to another thread to commit the modification there. Dieter
Jeff Youel wrote at 2003-10-7 11:10 -0700:
I've created an object to record some information (datetime, url, username) for each file downloaded from the site for later searching (CMF 1.3.1, Zope 2.6.1). To do this, I have a BTreeFolder with a private ZCatalog and put an new instance of a DownloadRecord object (my own object that just has string members holding the fields to be queried later) into the folder when the file is requested.
You could try to use "BTreeFolder2" and allow it to determine the Ids itself. Dieter
participants (3)
-
Dieter Maurer -
Jeff Youel -
Toby Dickenson