[Zope-dev] implementing zope.component 4.0

[Gary Poster]

On Nov 27, 2009, at 6:32 AM, Martijn Faassen wrote:

> Hi there,
> 
> Introduction
> ------------
> 
> So now that we've had some discussion and to exit the "bikeshed" phase, 

Wow.  That's abrupt, for something at the root of the entire stack.

I don't think long emails are very effective, but I'm not sure how else to reply to your long email.

> let's see about getting some volunteers to work on this.
> 
> The goal here is to make interfaces disappear into the language as much 
> as possible. This means that I'll ignore backwards compatibility while 
> sketching out the "ideal semantics" below - I have the impression we can 
> get consensus on the following behavior:
> 
> Simple adaptation:
> 
>   IFoo(adapted)
> 
> Named adaptation:
> 
>   IFoo(adapted, name="foo")
> 
> Adaptation with a default
> 
>   IFoo(adapted, default=bar)
> 
> Multi-adaptation:
> 
>   IFoo(one, two)
> 
> Named multi adaptation:
> 
>   IFoo(one, two, name="foo")
> 
> Multi-adaptation with a default:
> 
>   IFoo(one, two, default=bar)

I am in favor of the above, given a backwards compatibility story that makes existing packages work.

> 
> Utility lookup:
> 
>   IFoo()
> 
> Named utility lookup:
> 
>   IFoo(name="foo")
> 
> Utility lookup with a default:
> 
>   IFoo(default=bar)

I disagree with this.  More below.

> Where "name" and "default" can be combined. The name and default keyword 
> parameters have to be used explicitly - *args is interpreted as what to 
> adapt only. Any other keyword parameters should be rejected.
> 
> Utility lookups versus adapter lookups
> --------------------------------------
> 
> There was some discussion on whether utility lookups are really 
> something fundamentally different than adaptation as adaptation 
> *creates* a new instance while utility lookup uses a registered 
> instance. I think the essential part here is however: "give me an 
> instance that implements IFoo", and utility lookup fits there. We could 
> even envision a way to create utilities that *does* instantiate them on 
> the fly - it shouldn't affect the semantics for the user of the utility.

As above, I disagree.

As a matter of mechanics, when you register something we call an adapter, it is a callable that takes one or more arguments.  If we were going to follow the pattern that Marius laid out to establish what happens when, then we have this, roughly:

register callable that takes two arguments:
IFoo(bar, baz)

register callable that takes one argument:
IFoo(bar)

register callable that takes no arguments:
IFoo()

If instead we have the last step as what is proposed here

register non-callable
IFoo()

then I think that breaks an important pattern for usage understandability.

That is, IFoo() can have a semantic if that is valuable, but it is not the same as registering and getting non-called singletons.

Two by-the-ways:

1) The term "adapter" is a barrier to understandability, in my interviews.  This is particularly the case when people are introduced to the idea of "multiadapter" and "supscription adapter".  In what ways are these anything like a type cast?  IMO, they are not.  Our usage of adapter is as a factory.  Yes, it can be used in other ways--so can a Python class--but that is the essence of how our community uses this technology.  Calling all these ideas "adapters" accomplishes nothing.  Explaining all of the ideas as "a factory to produce an object that provides the interface" cleanly describes our usage, and both "adapters" and "multiadapters".

(To be complete, I am in favor of ditching "subscription adapters" in favor of other mechanisms related to named singleton lookups.)

One reason I like the syntax proposals for the adapter change is that they treat the interfaces as pluggable factories.  This is apt.

2) The term "utility" is another barrier to understandability.  They are singletons.  Explaining them as such is a "well, why didn't you say so" experience.

Therefore, I am in favor of removing the necessity to use the word utility.  That said, they are not factories.  They should not be mixed with the two.  My preference for future changes is to have an API using the ``singleton`` name.  Moreover, I think that some of the use cases that Marius referred to for underpowered "utilities" coud be remedied by having a utility/singleton lookup that allowed looking up by required values like the adapter/factory lookup.

> Features off the table for now
> -------------------------------
> 
> Saying an interface is implemented by a class (Python 2.6 and up) with a 
> decorator we'll leave out of the discussion for now.
> 
> It would also be come up with an improved API to look up the adapter 
> *before* it is called, but I'd also like to take this off the table for 
> this discussion.

It seems to me that this, along with the documentation call that Chris gave, is a much more valuable immediate effort.  One of the biggest complaints I heard was with debugging.  I've spent some thought on the debugging story, and have some APIs sketched out in my experiments--it was one of the first things I worked on.  To do it cleanly (the way I envision) would require some work, but a first cut wouldn't be too bad.

> Backwards compatibility
> -----------------------
> 
> Now let's get back to my favorite topic in this discussion: backwards 
> compatibility. The ideal semantics unfortunately break backwards 
> compatibility for the single adapter lookup case, as this supports a 
> second argument, the default.
> 
> The challenge is therefore to come up with a way to support the new 
> semantics without breaking the old.
> 
> We could introduce the following upgrade pattern:
> 
> zope.component 3.8.0: old semantics
> 
> zope.component 3.9: old semantics is the default. new semantics 
> supported too somehow but explicitly triggered.
> 
> zope.component 4.0: new semantics is the default. Old semantics is not
> supported anymore.
> 
> We could, if needed, maintain zope.component 3.x in parallel with the 
> new-semantics 4.0 line for a while.
> 
> A per-module triggering of the new semantics might be done like this:
> 
> from zope.component.__future__ import __new_lookup__
> 
> Is that implementable at all however? Someone needs to experiment.

I think it might work with frame tricks.  As I said, I'm worried about speed, but an experiment could clear that up.

I share Baiju's dislike of inventing __*__ names.  What is the necessity?  At least __future__ has precedence, I suppose, but Python devs have expressed their opinion clearly now that __*__ is theirs, and I think we should respect it in upcoming decisions.

Finally, per Martin's points, I'm not sure zope.component can actually ever deprecate the old spelling, so I'm not sure __future__ has the right semantic.  This is really __alt__ or something, IMO.

Alternatively, there's Shane's suggestion of a convenience wrapper.  I was thinking about that approach too.  It has obvious usability disadvantages over having the preferred API immediately available.

> Alternatively we could do something special when we see this: IFoo(foo, 
> bar). This is ambiguous - is the new semantics in use or the old one? If 
> the adapter cannot be looked up using multi adaptation we *could* fall 
> back on single adaptation under the assumption that the old semantics 
> are desired. But this will lead to a problem if the new semantics *was* 
> desired but the component simply could not be found.
> 
> I think it's important not to do a "big bang" upgrade but instead allow 
> people to upgrade bit by bit. It should be possible to compose an 
> application that mixes code that expects the old semantics with code 
> that expects the new semantics. A bit by bit upgrade I think would 
> ideally be on a per-module basis. I think it's important to make sure we 
> can support such an upgrade *before* we release any of this.

*Any* kind of backwards incompatibility is a huge deal.  I don't find that the simple improvements described here merit any kind of backwards incompatibility.  IMO, there needs to be a more compelling argument for that.

(To be clear, I'm not sure my experiments will turn up changes that merit backwards incompatibility!)

The zope.component API is done, I think.  We can experiment with other tools that provide a support shim for it, but breaking it is pretty questionable.

> 
> Conclusions
> -----------
> 
> Are people okay with the proposed semantics?

Only in part, as above.

> Would people be okay with such an upgrade path? Any better ideas?

The "future" thing might work.  Needs experiments, as you said.

> Most importantly, any volunteers?

While I like your desire to move forward, in this particular case I think this speed is rash.  I'd also prefer work on debugging tools before these syntax changes, but that's the kind of thing driven by who wants to do the work.

Gary