[ZODB-Dev] BTree set question

[John Belmonte]

Tim Peters wrote:
> A pertinent example of the latter is the case you brought up:  two
> transactions add the same key to the same set.  In what objective semantic
> sense is that "a conflict"?  Viewed as a mathemetical object, both
> transactions leave the set with the same value, and *on its own* that can't
> hurt.  It's possible that whenever a transaction adds a key to this set, in
> the same transaction it also performs other mutations related to that, and
> which aren't idempotent (== not harmless to do *those* twice).  The current
> implementation appears to be guarding against that case.  I don't know
> why -- or why not.

I think the rules should be conservative.  If a conflict comes up where 
the application really didn't require it, it can always retry.  The 
rules are conservative, with the notable exception of changing an 
existing key to the same value not being a conflict.

> OTOH, it's not guarding against possible bad consequences of two
> transactions deleting different keys from the same set.  For example, maybe
> it's a set of assets, and a "liquidate some assets to raise some cash"
> process tries to liquidate the smallest subset of assets just sufficient to
> raise a specific amount of cash, but subject to a global invariant that the
> total value of assets remaining must not fall below some fixed floor value.
> Then two concurrent transactions of this kind could easily liquidate
> different subsets of assets, and maintain the minimum value on their own,
> but end up falling below the floor when they're combined "by magic" via
> conflict resolution:  deleting different keys isn't *necessarily* safe.
> Neither is adding distinct keys.

I think your example is bad design in that the "total assets" value 
should not be implied.  It should be a scalar with conflict resolution, 
which can merge increments and decrements, and will raise a conflict 
value when falling below the floor value.  Any time you modify the asset 
set, you must also adjust the total assets value accordingly. 
Therefore, two transactions deleting different keys from the asset set 
is not a problem, and if total assets becomes too low there will be a 
conflict.  I use this kind of pattern throughout my application.

> In short, any fixed collection of BTree conflict resolution rules is going
> to be unsafe and/or needlessly inefficient for some applications.  I can't
> find an "organizing principle" consistent with all the choices currently
> made, so it's hard to explain (short of exhaustive enumeration) -- but since
> apps have relied on these undocumented behaviors for years, any change is
> dangerous.  Making conflict resolution strategies pluggable remains
> possible.

I don't see how having conservative resolution rules could be unsafe. 
So the question is, is upgrading the "changing an existing key to the 
same value" case to a conflict error going to bring someone's existing 
app to a crawl?

-John


-- 
http:// if  ile.org/