[ZODB-Dev] Ordering before commit hooks

Wed Aug 31 20:06:11 EDT 2005

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

ok this time I'm giving up. Not because you convinced me but because I
don't want to spend more energy on this since you'll have the last word
anyway.

You want to remove it, fine do it. Let's have several specific hook
ordering implementations on every frameworks on top of ZODB for the best...

Cheers,

	J.

P.S : Thanks Dieter fot the support.

Jim Fulton wrote:
> Dieter Maurer wrote:
> 
>> Jim Fulton wrote at 2005-8-31 10:20 -0400:
>>
>>> ...
>>>
>>>> Thus my proposal could be seen as well as a request to change (or to
>>>> make more flexible) the policy already in place. (Which was my use
>>>> case)
>>>
>>>
>>> Except that, as Tim and I have pointed out, the order gimmic didn't
>>> deal at all well with the use case of making something go last.
>>
>>
>>
>> Did you read my message?
> 
> 
> Yes.  Did you read my response> It was in the same note
> I wrote to Julien, so you probably missed it. Sorry.
> 
>>   The "order gimmic" provides a global scale along which
>>   an *integrator* can align hooks according to their
>>   relative needs (if they do not conflict altogether).
>>
>>   He (the integrator!) assigns orders to all hooks and can
>>   let individual hooks go at the start, the end, in the middle
>>   whereever it is necessary -- in a an (quite) easy and
>>   understandable way (the "import order" of packages might
>>   be an approximation -- but far less understandable and flexible
>>   and more error prone).
> 
> 
> To support this, you, will need some ZCML directives.
> building an add-on that provides a registry for ordered
> callbacks is a very small effort in comparisn.
> 
>>
>> I am starting to get a bit tired with this discussion.
> 
> 
> Me too.
> 
>>   I live since a long time with enhanced versions of Zope/ZODB.
>>   This can stay so, if improvements take too much effort
>>   to get them into the core.
> 
> 
> I'm sorry to hear that.  There are a number of reasons for this.
> I'm not trying to create obsticals.  I am extremely interested
> in getting ZODB to be much cleaner.  We are chipping away at
> making ZODB simpler and easier to understand.  For that reason,
> I'm gaurding the APIs pretty aggressively.
> 
> I'm happy to see contributions coming in.  The initial callback
> capability was contributed, and I'm glad to have it.  The fairly
> large contribution of Blob support is also very welcome.
> 
>>   Another alternative would be to publish a standard extension
>> package, that
>>   provides the (trivial) ordering.
>>   The only difficulty with this: It provides a *global*
>>   coordination facility; it would need to be available
>>   in all systems that anticipate a hook coordination need.
>>   The only advantage the ZODB has wrt. this solution:
>>   It is available everywhere.
> 
> 
> Only because it would be shared API. An add-on could be available
> everywhere too. It just needs to be published.  I don't even object
> to this being published with Zope if people do indeed find it useful.
> 
>>
>>> ...
>>>
>>>> Well yes and no. What if I want to go first instead ? Or in the middle
>>>> of 2 independently (external to my system) defined hooks ?
>>>
>>>
>>> What if you want one callback to go immediately after another?  What
>>> if you want a callback to be called only if another wasn't called (or
>>> was called). These are also legitimate policies that the order gimmick
>>> doesn't address.  They are policies that can be handled by a
>>> higher-level
>>> plug-in.
>>
>>
>>
>> That's true -- apart from the fact that you have not yet shown
>> the real life use cases for these hypothetical policies.
> 
> 
> Actually, no one has shown real life use cases for ordering.
> The only real use case I've seen is for going last, and the
> order gimmick doesn't support it well.
> 
>> It appears to me that these policies entail significant knowledge
>> among the various hooks about themselves. Such cases may
>> need specific solutions.
> 
> 
> Exactly.
> 
>>
>> Our use cases (at least mine) for the "order gimmick" come from a
>> common root: the wish to perform a summary operation once before
>> the commit rather then partial operations over and over again
>> during the transactions: e.g.:
>>
>>   *  one summary SQL UPDATE for the complete modified record
>>      rather than an SQL UPDATE for each attribute modification
>>
>>   *  one summary "index_object" rather than lots
>>      of "index_object" calls for each individual modification.
> 
> 
> These are valid use cases. This is why we added the callback
> mechanism. Neither of these require the order gimmick.
> 
>> The have order requirements (they should come after all possible
>> modifications that should go into the summary operation) but
>> they are not too strong.
> 
> 
> Ordering doesn't help this at all.  The callback mechanism
> itself provides the ability to execute special operations,
> like indexing after all normal operations.
> 
> If you think that callback operations might cause additional
> changes that might cause indexing, then you can try to get the
> indexing to go last, but ordering gimmick dosn't do that.
> 
> 
>> The "order gimmick", while not universal, provides an easy
>> and understandable facility to handle a large group of such
>> use cases.
> 
> 
> Huh? It doesn't help these use cases at all.
> 
>>
>>> ...
>>>
>>>> Jim, honestly where is the complication ?
>>>
>>>
>>> The API gets an extra argument who's use is far from clear.  People
>>> reading the API would rightly wonder why they would use it.
>>
>>
>>
>> Ordering the execution of hooks is not that unfamiliar.
>> Doing it by means of numbers, neither...
> 
> 
> It's not familiar to me.  It's also not effective.  I made the mistake
> of using this hack in zope.configure and it obfuscated the code.
> All I wanted was to cause certain actions to be executed after
> normal non-delayed actions. An API that allowed me to say that
> would have been clearer than what I did. (order=9999999).
> 
>>
>>> What
>>> does it gain them?
>>
>>
>>
>> There are few developpers who will use the hook, many hooks
>> will not need an explicit ordering. A developper who has
>> specific needs with guess what "order" is for ;-)
> 
> 
> That's what I'm afraid of. The gimmick will encourage code
> that missuses order.  There will be snippets of code that
> set an order. The reader of the code won't know if they thought
> the intent was to go last, or to fit into some mysterious
> non-obvious integer scheme. For example Julien used 100 to spell
> "last". Reading the code, I never would have guessed that.
> I would be wondering where to find the code with an order of
> 200. :)
> 
>>
>>> If they wanted something to go last (or first),
>>> what argument should they provide?
>>
>>
>>
>> Contrary to your opinion, hooks will probably rarely have the
>> need to be called as the very first or the very last one.
>> Usually, the requirements will be: after I was executed
>> some (hook) specific things should not happen anymore (because,
>> otherwise, I would need to run again).
> 
> 
> You might be right, but the only credible use case I've seen
> is to go last.  The example Julien has something going first
> and something going last.  Looking at the code, I don't see
> why the thing he wanted to go first needed to do so.
> The example you gave above had subscribers that wanted to go last.
> What evidence to I have of other orderings?  Julien did give
> a "future need" to order two specific systems.  IMO, having
> one system implement that requirement explicitly by invoking
> the other would be much clearer than having the
> requirement implemented implicitly through integers orders.
> 
>>
>>> ...
>>> Right. You need to use a common API. So define one and get people to use
>>> it. You *want* the common API to be defined by ZODB.
>>
>>
>>
>> Indeed, because then (almost) all ZODB users with hook synchronization
>> needs can use it.
> 
> 
> All people with hook synchronization needs can use an add on. It doesn't
> need to be in ZODB for them to do this.
> 
>>
>>> That's too low level.
>>
>>
>>
>> The amount of "common components" available for the resource
>> *coordination*
>> of independently developed packages is not so large.
>>
>>
>> If it does not go into the ZODB, then almost surely, competing
>> solutions will spring up that locally (e.g. for CPS, Plone, MySelf,
>> Archetypes,
>> ...) will do the job, causing pain when packages from these different
>> worlds
>> are integrated.
> 
> 
> Then propose a common framework.  I have no problem with a common
> framework. Put it in Zope if you want. It doesn't belong
> in ZODB.
> 
>>
>> It would be nice, if a common solution could be established
>> at or below CMF (as most of the worlds above are above it) -- but
>> after this discussion, I fear, it will not happen...
> 
> 
> Why? (That was rhetorical :) There are lots of layers above ZODB
> where this can go.
> 
> Jim
> 

- --
Julien Anguenot | Nuxeo R&D (Paris, France)
CPS Platform : http://www.cps-project.org
Zope3 / ECM   : http://www.z3lab.org
mail: anguenot at nuxeo.com; tel: +33 (0) 6 72 57 57 66
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iD8DBQFDFkXzGhoG8MxZ/pIRAgC0AKCFljJ/ROHe0zHKXRChrsJZjnPDDQCcDj/5
G1HDRAgRDxh/cZnHIDzmcdc=
=U/PG
-----END PGP SIGNATURE-----