Am 20.05.2010 20:04, schrieb robert rottermann:
Am 20.05.2010 19:34, schrieb Maik Derstappen, Derstappen IT:
Am 20.05.2010 16:05, schrieb Jürgen Herrmann:
On Thu, 20 May 2010 15:59:58 +0200, Patrick Gerken <do3ccqrv@googlemail.com> wrote:
On Thu, May 20, 2010 at 14:18, Jürgen Herrmann <Juergen.Herrmann@xlhost.de>wrote:
hi there!
as our zope2 zodb grows we experience more and more conflicts with long running transactions. so i'm going to rewrite these methods to commit transactions in batches - here my attempt to tackle this problem. the idea is to get a list of objects to be modified in this transaction, break it down to batches and commit after each batch was processed. to avoid zope's retry-on-conflict machinery and still have a chance to retry per batch transaction commits, the iteration over one batch is guarded by a try:except ConflictError: and a retry logic.
Zope already has its own retry mechanism, maybe you might want to call your long running method multiple times, each time working on the next batch.
in fact zope's retry machinery is kicking in too often which is a bad thing if the retried transaction runs for say 30min or so.
having to gather the list of objects outside of zope, doing the batching there and calling into zope multiple times - i wanted to avoid that.
i'd like to have all the logic contained in one zope method for the sake of simplicity on the caller's side (most of these are kicked off by cron jobs).
jürgen
take a look at http://pypi.python.org/pypi/affinitic.zamqp maybe it is a solution for you. You can deligate your jobs out of zope and get it back after some time. zamqp then fire an event and you subscribe this event to handle thie results.
with kind regards Maik Derstappen
maik, this looks interesting. just out of curiosity: how would one alert the user that a long running job (say after some 10 minutes) has finished. does the client have to poll? or is there some "magic" push functionality?
robert
hi robert, zamqp has an process that continuously receive new message from his message queue and call a callback method which fire the event, that you can subscribe. see zamqp and carrot docs for more info ;) regards maik -- ======================================================================== Derstappen I T Consulting Tel: +49 ( 341 ) 600 13 0 31 Zope/E-Mail/Backup/Monitoring Mobil: +49 ( 178 ) 861 2 833 M a i k D e r s t a p p e n Fax: +49 ( 180 ) 5 021 121 90 56 H e r l o ß s o h n s t r 12 Email: maik.derstappen@derstappen-it.de 0 4 1 5 5 L e i p z i g Internet: http://www.derstappen-it.de ========================================================================