On Thu, 20 May 2010 18:00:43 -0600, Shane Hathaway <shane@hathawaymix.org> wrote:
On 05/20/2010 06:18 AM, Jürgen Herrmann wrote:
def myLongRunningMethod(self):
BATCH_SIZE = 10 MAX_CONFLICTS = 3
work_items = [some, work, to , do, ...] # list of persistent objects to be modified in this method
idx = 0 while idx< len(work_items): conflicts = 0 try: my_batch = work_items[idx, idx+BATCH_SIZE] for work_item in my_batch: do_some_work(work_item) transaction.commit() except ConflicError: conflicts += 1 if conflicts> MAX_CONFLICTS: raise else: idx += BATCH_SIZE
does this sound like a reasonable approach?
More than that, it looks nearly finished. ;-) The transaction machinery
does all the heavy lifting, so you don't need to do much. Issues to fix:
- Your code resets the "conflicts" counter in every iteration. You obviously didn't intend that. hmm, why not? every batch should be retried up to MAX_CONFLICTS times?
- You should start each loop with "transaction.begin()" to avoid any side effects of transactions started before the long running method was called. cool.
- Writing thorough tests for this kind of code is very important. If you don't, it might be a while before you discover that ConflictError was misspelled. actually python wouldn't compile that code above as ConflicError must resolve to a class at compile time but i get your point, agreed :)
Shane
thanks for your answers and have a nice day everbody. jürgen --
XLhost.de - eXperts in Linux hosting ® <<
XLhost.de GmbH Jürgen Herrmann, Geschäftsführer Boelckestrasse 21, 93051 Regensburg, Germany Geschäftsführer: Volker Geith, Jürgen Herrmann Registriert unter: HRB9918 Umsatzsteuer-Identifikationsnummer: DE245931218 Fon: +49 (0)800 XLHOSTDE [0800 95467833] Fax: +49 (0)800 95467830 WEB: http://www.XLhost.de IRC: #XLhost@irc.quakenet.org