[Zope] long running methods

[Maik Derstappen, Derstappen IT]

Am 20.05.2010 16:05, schrieb Jürgen Herrmann:
> On Thu, 20 May 2010 15:59:58 +0200, Patrick Gerken
> <do3ccqrv at googlemail.com>  wrote:
>
>> On Thu, May 20, 2010 at 14:18, Jürgen Herrmann
>> <Juergen.Herrmann at xlhost.de>wrote:
>>
>>
>>> hi there!
>>>
>>> as our zope2 zodb grows we experience more and more conflicts
>>> with long running transactions. so i'm going to rewrite these
>>> methods to commit transactions in batches - here my attempt
>>> to tackle this problem. the idea is to get a list of objects
>>> to be modified in this transaction, break it down to batches
>>> and commit after each batch was processed. to avoid zope's
>>> retry-on-conflict machinery and still have a chance to retry
>>> per batch transaction commits, the iteration over one batch
>>> is guarded by a try:except ConflictError: and a retry logic.
>>>
>>>
>> Zope already has its own retry mechanism, maybe you might want to
>> call your long running method multiple times, each time working on the
>> next batch.
>>
>>
> in fact zope's retry machinery is kicking in too often which is a
> bad thing if the retried transaction runs for say 30min or so.
>
> having to gather the list of objects outside of zope, doing the
> batching there and calling into zope multiple times - i wanted
> to avoid that.
>
> i'd like to have all the logic contained in one zope method
> for the sake of simplicity on the caller's side (most of these
> are kicked off by cron jobs).
>
> jürgen
>
take a look at http://pypi.python.org/pypi/affinitic.zamqp maybe it is a 
solution for you.
You can deligate your jobs out of zope and get it back after some time.
zamqp then fire an event and you subscribe this event to handle thie 
results.

with kind regards
Maik Derstappen

-- 
========================================================================

Derstappen    I T  Consulting   Tel:         +49 ( 341 )   600  13  0 31
Zope/E-Mail/Backup/Monitoring   Mobil:       +49 ( 178 )   861  2    833
M a i k   D e r s t a p p e n   Fax:         +49 ( 180 ) 5 021 121 90 56
H e r l o ß s o h n s t r  12   Email:  maik.derstappen at derstappen-it.de
0 4 1 5 5       L e i p z i g   Internet:    http://www.derstappen-it.de

========================================================================