[Zope] long running methods

[Shane Hathaway]

On 05/20/2010 06:18 AM, Jürgen Herrmann wrote:
> def myLongRunningMethod(self):
>
>    BATCH_SIZE = 10
>    MAX_CONFLICTS = 3
>
>    work_items = [some, work, to , do, ...]
>    # list of persistent objects to be modified in this method
>
>    idx = 0
>    while idx<  len(work_items):
>      conflicts = 0
>      try:
>        my_batch = work_items[idx, idx+BATCH_SIZE]
>        for work_item in my_batch:
>          do_some_work(work_item)
>        transaction.commit()
>      except ConflicError:
>        conflicts += 1
>        if conflicts>  MAX_CONFLICTS:
>          raise
>      else:
>        idx += BATCH_SIZE
>
> does this sound like a reasonable approach?

More than that, it looks nearly finished. ;-)  The transaction machinery 
does all the heavy lifting, so you don't need to do much.  Issues to fix:

- Your code resets the "conflicts" counter in every iteration.  You 
obviously didn't intend that.

- You should start each loop with "transaction.begin()" to avoid any 
side effects of transactions started before the long running method was 
called.

- Writing thorough tests for this kind of code is very important.  If 
you don't, it might be a while before you discover that ConflictError 
was misspelled.

Shane