[Zope] long running methods

[Jürgen Herrmann]

On Thu, 20 May 2010 18:00:43 -0600, Shane Hathaway <shane at hathawaymix.org>
wrote:
> On 05/20/2010 06:18 AM, Jürgen Herrmann wrote:
>> def myLongRunningMethod(self):
>>
>>    BATCH_SIZE = 10
>>    MAX_CONFLICTS = 3
>>
>>    work_items = [some, work, to , do, ...]
>>    # list of persistent objects to be modified in this method
>>
>>    idx = 0
>>    while idx<  len(work_items):
>>      conflicts = 0
>>      try:
>>        my_batch = work_items[idx, idx+BATCH_SIZE]
>>        for work_item in my_batch:
>>          do_some_work(work_item)
>>        transaction.commit()
>>      except ConflicError:
>>        conflicts += 1
>>        if conflicts>  MAX_CONFLICTS:
>>          raise
>>      else:
>>        idx += BATCH_SIZE
>>
>> does this sound like a reasonable approach?
> 
> More than that, it looks nearly finished. ;-)  The transaction machinery

> does all the heavy lifting, so you don't need to do much.  Issues to
fix:
> 
> - Your code resets the "conflicts" counter in every iteration.  You 
> obviously didn't intend that.
hmm, why not? every batch should be retried up to MAX_CONFLICTS times?
> 
> - You should start each loop with "transaction.begin()" to avoid any 
> side effects of transactions started before the long running method was 
> called.
cool.
> 
> - Writing thorough tests for this kind of code is very important.  If 
> you don't, it might be a while before you discover that ConflictError 
> was misspelled.
actually python wouldn't compile that code above as ConflicError must 
resolve to a class at compile time but i get your point, agreed :)
> 
> Shane

thanks for your answers and have a nice day everbody.

jürgen
-- 
>> XLhost.de - eXperts in Linux hosting ® <<

XLhost.de GmbH
Jürgen Herrmann, Geschäftsführer
Boelckestrasse 21, 93051 Regensburg, Germany

Geschäftsführer: Volker Geith, Jürgen Herrmann
Registriert unter: HRB9918
Umsatzsteuer-Identifikationsnummer: DE245931218

Fon:  +49 (0)800 XLHOSTDE [0800 95467833]
Fax:  +49 (0)800 95467830

WEB:  http://www.XLhost.de
IRC:  #XLhost at irc.quakenet.org