[ZODB-Dev] Automating retry management

[Jim Fulton]

On Tue, May 11, 2010 at 11:35 AM, Laurence Rowe <l at lrowe.co.uk> wrote:
> On 11 May 2010 15:08, Jim Fulton <jim at zope.com> wrote:
>> On Tue, May 11, 2010 at 8:38 AM, Benji York <benji at zope.com> wrote:
>>> On Tue, May 11, 2010 at 7:34 AM, Jim Fulton <jim at zope.com> wrote:
>>>> [...] The best I've been
>>>> able to come up with is something like:
>>>>
>>>>    t = ZODB.transaction(3)
>>>>    while t.trying:
>>>>        with t:
>>>>            ... transaction body ...
>>>
>>> I think you could get this to work:
>>>
>>> for transaction in ZODB.retries(3):
>>>    with transaction:
>>>        ... transaction body ...
>>>
>>> ZODB.retries would return an iterator that would raise StopIteration on
>>> the next go-round if the previously yielded context manager exited
>>> without a ConflictError.
>>
>> This is an improvement. It's still unsatisfying, but I don't think I'm going to
>> get satisfaction. :)
>>
>> BTW, if I do something like this, I think I'll add a retry exception to
>> the transaction package and have ZODB.POSException.ConflictError
>> extend it so I can add the retry automation to the transaction package.
>
> The repoze.retry package lets you configure a list of exceptions.
> http://pypi.python.org/pypi/repoze.retry
> http://svn.repoze.org/repoze.retry/trunk/repoze/retry/__init__.py
>
>  Though it seems inspecting the error text is required for most sql
> database errors to know if they are retryable, as ZPsycoPGDA does:
>
>  188                 except (psycopg2.ProgrammingError,
> psycopg2.IntegrityError), e:
>  189                     if e.args[0].find("concurrent update") > -1:
>  190                         raise ConflictError
>
> (https://dndg.it/cgi-bin/gitweb.cgi?p=public/psycopg2.git;a=blob;f=ZPsycopgDA/db.py)
>
> For PostgreSQL it should be sufficient to catch these errors and raise
> Retry during tpc_vote.
>
> For databases which do not provide MVCC in the same way as PostgreSQL,
> concurrency errors could be manifested at any point in the
> transaction. Even Oracle can raise an error during a long running
> transaction when insufficient rollback space is available, resulting
> in what is essentially a read conflict error. Such errors could not be
> caught by a data manager and reraised as a Retry exception.
>
> I think it might be useful to add an optional method to data managers
> that is queried by the retry automation machinery to see if an
> exception should potentially be retried. Perhaps this would best be
> accomplished in two steps:
>
> 1. Add an optional property to data managers called ``retryable``.
> This is a list of potentially retryable exceptions. When a data
> manager is added to the transaction, the transaction's list of
> retryable exceptions is extended by the joining data managers list of
> retryable exceptions.
>
> t = transaction.begin()
> try:
>    application()
> except t.retryable, e:
>    t.retry(e):
>
> 2. t.retry(e) is then checks with each registered data manager if that
> particular exceptions is retryable, and if so raises Retry.
>
> def retry(self, e):
>    for datamanager in self._resources:
>        try:
>            retry = datamanager.retry
>        except AttributeError:
>            continue
>        if isinstance(e, datamanager.retryable):
>            datamanager.retry(e) # dm may raise Retry here

Thanks.

I don't think we need 1 and 2.
I'm inclined to go with 2.

Jim

-- 
Jim Fulton