Thomas Lotze wrote:

Martijn Faassen wrote:

...

Are people okay with the proposed semantics?

I am.

...

Would people be okay with such an upgrade path? Any better ideas?

I'm not comfortable with the idea of an automatic fall-back for IFoo(x, y) but maybe that changes after thinking about it some more.

I'm not comfortable with it either. I was just thinking out loud on that. My question should've been formulated more clearly. I mean an upgrade path where 3.x and 4.x are maintained in parallel and people can do an incremental upgrade in 3.x.

...

Most importantly, any volunteers?

I'd like to work on this.

Great! Regards, Martijn

Chris Withers wrote:

Martijn Faassen wrote:

...

Simple adaptation:

IFoo(adapted)

Is there an implied default of None here or would a ComponentLookupError be raised?

I'd say a ComponentLookupError. Doesn't it do that now? Anyway, I'd say if you want a default specify it, otherwise it'll give an error. Regards, Martijn

Baiju M wrote:

...

from zope.__future__ import new_adapter_lookup?

Let's leave how special names are created in Python to Python. We already have __parent__, __annotation__ etc.

What if Python brings a special name which we are using with a different semantics.

Python's not going to introduce a new keyword or __special__ method any time soon (especially given the recent moratorium on language changes), or break existing usages. I think a __future__ to import from would be pretty safe. Besides, hopefully soon enough we could remove all those imports again, as we're done upgrading.

May be we can use special prefix for Zope's own special names, some thing like: _z__specialname__ .

We're just following the Python pattern here, the from __future__ import pattern. I think: from zope.component.__future__ import new_lookup would clearly signal even to a completely unaware Python programmer that something special is going on. [Marius spelled it better than I did; no need to double underscore __new_lookup__] Regards, Martijn

On Fri, Nov 27, 2009 at 02:38:55PM +0100, Martijn Faassen wrote:

I'd ask people to think about this approach without considering backwards compatibility issues first. Especially given the goal "making component lookup disappear into the language" makes me think just all making it calling an interface would be the most elegant approach.

You're convincing. (I just can't force myself ignore backwards-compatibility issues, I suppose.)

So imagining we didn't have to worry about backwards compatibility, would you still propose that API, making a difference between adapter and utility lookup?

Yes.

What's the motivation?

The "utilities must be singletons" logic hardcoded in the ZCA. provideAdapter(factory, adapts=(one, two, three)) provideAdapter(factory, adapts=(one, two)) provideAdapter(factory, adapts=(one, )) The natural progression, to me, is provideAdapter(factory, adapts=()) rather than provideUtility(singleton) If we decide, ignoring BBB concerns, to make provideUtility(singleton, provides=IFoo) be equivalent to provideAdapter(lambda: singleton, adapts=(), provides=IFoo) then I'd be very happy to use IFoo() for utility lookup. This also assumes that I'm free to provideAdapter(arbitrary_callable, adapts=()) and use computed-utility-lookup, or even create utilities on demand in my arbitrary_callable. Three cheers for utility and empty-tuple-adapter unification!

And would you deprecate plain adapter calls and prefer 'adapt' all the time?

Ignoring BBB, and assuming utilities-are-just-adapters-on-empty-tuples, then I'd prefer to use your proposed IFoo(*args, **kw) syntax. The more I think about it, the more I like this solution. Ignoring BBB and assuming utilities are distinct fowl, I'd prefer to have just IFoo.adapt(*args, **kw) and IFoo.utility(**kw). The phrasing of your question ("deprecate") makes ignoring BBB impossible, in which case I'd prefer to have a 100% backwards-compatible IFoo(single_arg, [default]) + IFoo.adapt(*args, **kw) + IFoo.utility(**kw).

So as to prevent bikeshedding the API too much, I'm going to take your +0.5 anyway. :)

Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development

Marius Gedminas wrote: [snip]

The "utilities must be singletons" logic hardcoded in the ZCA.

provideAdapter(factory, adapts=(one, two, three)) provideAdapter(factory, adapts=(one, two)) provideAdapter(factory, adapts=(one, ))

The natural progression, to me, is

provideAdapter(factory, adapts=())

rather than

provideUtility(singleton)

If we decide, ignoring BBB concerns, to make

provideUtility(singleton, provides=IFoo)

be equivalent to

provideAdapter(lambda: singleton, adapts=(), provides=IFoo)

then I'd be very happy to use

IFoo()

for utility lookup.

This also assumes that I'm free to

provideAdapter(arbitrary_callable, adapts=())

and use computed-utility-lookup, or even create utilities on demand in my arbitrary_callable.

Three cheers for utility and empty-tuple-adapter unification!

This sounds appealing to me. Making it so should be seen as a separate project. The backwards compatibility implications are rather larger, especially in the face of persistent registrations.. Someone would need to experiment whether zope.component.registry can be refactored into these terms. Effectively we'd merge _utility_registrations into _adapter_registrations. But how would this impact the performance of things like 'registeredUtilities()', for instance? I wonder too what we could do to subscription_registrations and handler_registrations, I hadn't considered those in these discussions at all yet. Regards, Martijn

Adam GROSZER wrote:

I had a feeling that adapter lookup can be alone slowish with lots of registrations. We had a large project that was cut in half and the z3c.form UI, which is rather heavily adaptation based got a boost after that.

Interesting. It'd be interesting to do some experiments with this. Could you perhaps look into writing some kind of stress-test script? Regards, Martijn

Hello Hanno, Seems that that was an adapter heavy app. Was before the cut:

...

...

len(getGlobalSiteManager()._utility_registrations) 980 len(getGlobalSiteManager()._adapter_registrations) 1432 len(getGlobalSiteManager()._handler_registrations) 63 len(getGlobalSiteManager()._subscription_registrations) 50

After the cut:

...

...

len(getGlobalSiteManager()._utility_registrations) 739 len(getGlobalSiteManager()._adapter_registrations) 933 len(getGlobalSiteManager()._handler_registrations) 37 len(getGlobalSiteManager()._subscription_registrations) 33

So around 1/3 of the registrations was gone and that made a difference (well, with z3c.form generated forms). Would be interesting to profile how many adapter lookups are done by a simple z3c.form to render... Saturday, November 28, 2009, 2:32:50 PM, you wrote: HS> On Sat, Nov 28, 2009 at 12:14 PM, Martijn Faassen HS> <faassen@startifact.com> wrote:

...

Adam GROSZER wrote:

...

I had a feeling that adapter lookup can be alone slowish with lots of registrations. We had a large project that was cut in half and the z3c.form UI, which is rather heavily adaptation based got a boost after that.

HS> What is a "large project" in your case? Just as an example here's the HS> "size" of the global registry in a typical Plone project:

...

...

...

getGlobalSiteManager() HS> <BaseGlobalComponents base> len(getGlobalSiteManager()._utility_registrations) HS> 1091 len(getGlobalSiteManager()._adapter_registrations) HS> 1283 len(getGlobalSiteManager()._handler_registrations) HS> 139 len(getGlobalSiteManager()._subscription_registrations) HS> 3

...

Interesting. It'd be interesting to do some experiments with this. Could you perhaps look into writing some kind of stress-test script?

HS> I haven't done any real performance measurements but the various HS> zope.interface/component API's are among the top of every profile run HS> I do in Plone. HS> To me it looks like checking if an interface is provided by a context HS> is non-trivial and the main bottleneck in our case. The classes HS> underlying our typical contexts are pretty fat classes with a long HS> inheritance chain contributing dozens of interfaces. HS> The actual registry lookups seem to be rather fast, they should be HS> essentially dict lookups, which perform well with a dict size of just HS> about 1000. HS> On the other hand I noticed that z3c.form and it's use of the ZCA is HS> indeed much slower than stitching forms together via ZPT macros as HS> done for example by Archetypes. HS> Hanno HS> _______________________________________________ HS> Zope-Dev maillist - Zope-Dev@zope.org HS> https://mail.zope.org/mailman/listinfo/zope-dev HS> ** No cross posts or HTML encoding! ** HS> (Related lists - HS> https://mail.zope.org/mailman/listinfo/zope-announce HS> https://mail.zope.org/mailman/listinfo/zope ) -- Best regards, Adam GROSZER mailto:agroszer@gmail.com -- Quote of the day: The only way to amuse some people is to slip and fall on an icy pavement.

Martijn Faassen wrote:

...

+0.5 --- I can live with it. Backwards incompatibility with IFoo(one, default) will be a slight inconvenience. There were proposals I liked more (IFoo.adapt(), IFoo.utility()) and proposals I liked less (IFoo((one, two, we_like_parentheses, and_screw_people_adapting_tuples))).

I'd ask people to think about this approach without considering backwards compatibility issues first.

Well, I don't think the difference between adapters and utilities is important, but I can understand why some people find calling the interface odd: it is when you think about it objectively. We also have the backwards compatibility issue. Combine the two, and you get me suggesting: marker = object() class IInterface(Interface): def get(self,*to_adapt,default=marker,name=None): "Adapt whatever is passed in to this interface" to_adapt==() would imply a utility lookup, in the old way of thinking about things, but I really don't think that would be important. I'll chuck +sys.maxint as my suggestion, obviously, but +sys.maxint-1 at the "from zope.component.__future__ import new_lookup" and calling interfaces you've described otherwise. cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

Charlie Clark wrote:

Am 27.11.2009, 15:57 Uhr, schrieb Chris Withers <chris@simplistix.co.uk>:

...

Well, I don't think the difference between adapters and utilities is important, but I can understand why some people find calling the interface odd: it is when you think about it objectively.

I have to agree with this. IFoo(myobject) is not intuitive. I know it used a lot because it's convenient shorthand but I've never read anywhere that interface classes are, in fact, callables. We certainly don't normally treat them as such.

It's quite intuitive to me.. Compare it with plain python:

...

...

int(something)

or:

...

...

str(something) len(something)

You say "give me something that's an int for the argument", or "give me something that's a string for the argument". You don't care how it accomplishes it, as long as it gives the right value back. It's even like adapters in the following way:

...

...

int(1)

Gives back the object itself, as it already is an int.

...

...

int('1') int(1.5)

Int is also "registered for" strings and floats, but essential different styles of "adaptation" happen there. Calling an interface is really very similar to this. The main difference is that we don't use the concrete implementation's factory but that we use the interface that specifies the abstract behavior. That is a difference, but doesn't seem to be a huge step in my mind.

One of the things that I have grown to appreciate with the ZCA is the advantage of spelling out the relationship between objects and I'll happily take a little verbosity over magic.

It's not verbosity versus magic. It's a better API versus a worse API.

The discussion does highlight a key source of confusion about Zope interfaces: they are, at the same time, an object specification and a kind of name tag or token that objects can provide upon request. While I know that the second function is derived from the first it is conceptually different.

My preference, for the sake of clarity:

adapted = an_easy_way_to_the_registry.adapt(*objects_to_be_adapted, **identifiers)

I'm not sure how this is supposed to work; what is identifiers?

That adapters are all callable now seems to be an accepted convention, presumably from convenience. But my understanding of adapters does not imply this.

I hope to have shown to you above that my understanding of adapters does. Regards, Martijn

On Sat, Nov 28, 2009 at 16:39, Charlie Clark <charlie.clark@clark-consulting.eu> wrote:

So adapters are reduced to type conversion?

That's what adapters are. They aren't reduced to it, it's what they do. They adapt one object with one interface to have another interface. That can indeed be seen as a type conversion.

Thanks for the comparison but it is semantically so different and interfaces can be used for things other than adapters that I disagree.

Yes but adapters can only be used as adapters, and the topic was the syntax for adapting an object.

The most common example I know of the syntax is with INameChooser() which brings us back to the differences (real or imaginary) between utilities and adapters.

I agree that calling an interface like that is a strange thing to do. I don't know what that would do, even. I have however never ever seen that done.

It's quite likely that I'm wrong in this but I see great potential using adapters for delegation rather than straight conversion.

But the delegation is adaptation. There is no difference.

I have very much come to appreciate the power of this delegation in, say, BrowserViews; even if it did take me several months to understand the multiadapter pattern!

I hear this a lot, so this is apparently something that is common to take a while to grasp. Any ideas of what to make multi-adapters more understandable would probably be a good thing. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64

Chris McDonough wrote:

Lennart Regebro wrote:

...

...

I have very much come to appreciate the power of this delegation in, say, BrowserViews; even if it did take me several months to understand the multiadapter pattern! I hear this a lot, so this is apparently something that is common to take a while to grasp. Any ideas of what to make multi-adapters more understandable would probably be a good thing.

The typical misunderstanding starts like this, I think: [snip scenario]

Personally, even I don't really know how it works.

I think this scenario is actually a lot more common among those of us (you and me) who *do* have an idea of what multi adaptation actually does. :) I think what you are describing is a lack of understanding of the detailed mechanism. Perhaps it's different for you, but I haven't heard stories of beginners being confused by this. I think you only tend to run into this if you do something quite advanced with the ZCA, i.e. build frameworky things yourself. That said, better documentation would again be useful. Regards, Martijn

On Sat, Nov 28, 2009 at 17:35, Charlie Clark <charlie.clark@clark-consulting.eu> wrote:

Am 28.11.2009, 16:55 Uhr, schrieb Lennart Regebro <regebro@gmail.com>:

...

That's what adapters are. They aren't reduced to it, it's what they do. They adapt one object with one interface to have another interface. That can indeed be seen as a type conversion.

I agree that that is probably the most common use for them.

No, that's what they are. An adapter is something that adapts one interface to another. That's the definition of the adapter pattern. It's not the most common usecase, thats THE usecase. If it doesn't do that, it's not an adapter.

My point is you can't necessarily tell much from the interface name.

Of course.

...

I agree that calling an interface like that is a strange thing to do. I don't know what that would do, even. I have however never ever seen that done.

It's utility for avoid name conflicts when adding a new object to a container:

 from zope.container.interfaces import INameChooser name = INameChooser(container).chooseName(obj.getId(), obj)

Ah, OK. You skipped the container parameter. That's just an adapter.

...

But the delegation is adaptation. There is no difference.

For me, conceptually, there is quite a difference. An adapter is closely coupled to the object it is adapting.

Absolutely.

The common example is the headphone jack - electric impulses in and out. Delegation, at least the way I think of it, places the emphasis on separating the functionality, ie. decoupling, of the adapter from the object being adapted, even though technically it's exactly the same.

Well, that's just a matter of creating the Interface with a specific goal in mind. You separate two types of adapters in your inner conceptualisation: Case 1. I have an American Power port, but I need to put it into a european one, so I get an adapter. Case 2. I'm going to make a laptop power supply. I therefore design it so that I can connect various adapters into the power supply, MacBook adapter style. In the first case I have two interfaces designed by others, and adapt on to the other. In the second case, I design an interface specifically so it will be easy to make adapters for it to the different uses I envision. But those cases are still just adapters. It's different situtions to be in as a programmer, but technically they are exactly the same. It's just adapters.

I think of adapters as the gadgets that James Bond gets given by "Q" - it helps me get away from thinking about implementation.

I don't see how that works.

hm, good question. With BrowserViews I think the problem is possibly with the whole idea of the REQUEST object which it's easy to oversee and just treat like a dictionary or storage. I normally concentrate on emphasising that the view has been delegated presentational responsibility by the content object in a particular context (maybe it is easier to understand adapting the content object and the request for a piece of HTML or PDF, etc.). Would it be too far fetched to imagine an engine adapting a car and petrol to provide motion?

Works for me. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64

Charlie Clark wrote: [snip]

So adapters are reduced to type conversion?

Adaptation is "give me something that provides this API for this object". Conversion in Python asks the same. Adaption just formalizes this and generalizes it. I don't see how it's a reduction.

...

Calling an interface is really very similar to this. The main difference is that we don't use the concrete implementation's factory but that we use the interface that specifies the abstract behavior. That is a difference, but doesn't seem to be a huge step in my mind.

Thanks for the comparison but it is semantically so different and interfaces can be used for things other than adapters that I disagree. The most common example I know of the syntax is with INameChooser() which brings us back to the differences (real or imaginary) between utilities and adapters.

I don't think it's that different at all semantically if you think of it. I think what you're getting at with the name chooser example is that adapters are not really used for conversion but for accessing a *feature* for an object. This was in fact an old proposed name for adapters in Zope 3. So, with INameChooser you'd like the name chooser feature for a container. And "int()" *can* be seen as wanting the integer feature for a particular string. But that's not as convincing as the example of len in Python. 'len()' asks for the size feature for an object (a list, a string, a dict, etc). The difference here is that with conversion, often the original value is considered to be unimportant anymore - once I have my integer I can forget my string. That's not the case with len - the original object is still there and relevant. With adaptation both patterns exist, but the feature pattern is more common. To step away from adaptation for a bit, I find utility lookups interesting to compare with imports in Python. The import statement in Python is used to import a single global instance of a particular thing (an instance, or a module instance). Implicitly the importing code expects the imported thing to fulfill a particular interface. A utility lookup does something very similar, except that the interface is made explicit and it's more easy to plug in alternatives. I've toyed around with the idea of turning utility lookup into imports: from foo.bar.baz import IFoo as foo would be the equivalent of: foo = component.getUtility(IFoo) But unfortunately this idea has some drawbacks: * how to handle named utilities and defaults? * I suspect it cannot be easily implemented at all. :) * most unfortunately, imports are usually done on module-level during import time while utility lookups *cannot* be done on module-level because during import time the utility registry is not initialized yet. So in fact we need to do this in two steps: import something for the utility during import time, and then during run time do the actual utility lookup. That's exactly what this would do: from foo.bar.baz import IFoo def main(): foo = IFoo() [snip]

It's quite likely that I'm wrong in this but I see great potential using adapters for delegation rather than straight conversion. I have very much come to appreciate the power of this delegation in, say, BrowserViews; even if it did take me several months to understand the multiadapter pattern!

Delegation is indeed a special property that conversion and feature patterns in plain Python don't have (unless I missed an example). The thing that is returned in plain Python is usually of a type that's so well known by the programmer it disappears into the background. With adapters this is less common. My proposal hopes to make some of these types appear into the background a bit more too, though.

Because I do, repeatedly, make simple mistakes with the adapter, utility (wrong name, wrong signature) stuff I very much appreciate attempts to simplify and clarify the API. But I will greet them the same poor grasp of the underlying concepts than I did the originals!

I agree that we should *also* work at explaining the underlying concepts more and better. Regards, Martijn

Martijn Faassen wrote:

Chris McDonough wrote:

...

Martijn Faassen wrote: [snip]

...

So now that we've had some discussion and to exit the "bikeshed" phase, let's see about getting some volunteers to work on this.

The goal here is to make interfaces disappear into the language as much as possible. I suggest we think to ourselves who the primary beneficiary of this goal is. I don't really disagree with anything you say here in absolute terms, but if the goal is increased adoption by new users, I think there are (relatively) more fundamental things that could be done to help.

For this measure I'm not interested in increased adoption by new users. I just want to be able to have a more convenient way to express these things. So, my goal is to make life easier for myself (and hopefully others).

Fair enough. - C

Tres Seaver wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Martijn Faassen wrote:

...

Are people okay with the proposed semantics?

+1.

...

Would people be okay with such an upgrade path? Any better ideas?

I would start issuign DeprecationWarnings (yes, I know I'm their worst fan, but we can't keep BBB here, so warnings are appropriate) for positional defaults in 3.9.x.

Good idea. Let's release a 3.9.x (or a 3.10) that does that as soon as possible.

I think we should also document the "don't call" API better (pure lookup): there have been use cases for this feature (e.g., "adapt to scalar / string") floating around for a *long* time now, unsupported. If that means accepting the zope.registry change Chris proposed, I'm fine with that.

I'm not sure I understand how Chris' zope.registry change has something to do with this API, perhaps I missed something? I agree that we could document this API better though.

...

Most importantly, any volunteers?

I can help some with this. Perhaps we should start by fleshing it *really good docs* (not doctests) for zope.component 4.0, including careful notes about how to make existing code compatible with both 3.x and 4.x APIs.

I'd be happy to help with this. I think that this work could be started in the trunk right away, as this documentation should be useful for 3.x as well, at least in terms of transitioning existing code. Regards, Martijn

Martin Aspeli wrote:

Martijn Faassen wrote:

...

Multi-adaptation:

IFoo(one, two)

Please note that this will break an incredible amount of code "in the wild". A good number of my packages do something like this:

foo = IFoo(context, None) if foo is None: ...

Yes, that this would break a lot of code is well known [backwards compatibility discussion]

-1

Because we now have so many packages and things are being released as eggs and mixed up in various platforms and projects, this type of "gross" backwards incompatibility is virtually impossible to manage.

To take an example, I'm sure Stefan & co will release z3c.form 3 depending on zope.component 4 before long, and we'll want to use that in Plone. Except we can't, because even if z3c.form never uses the IFoo(one, two) syntax, everything in Plone that uses IFoo(context, None) would suddenly break.

That's why I think it's important to have a: * a zope.component 3.x that supports both patterns * a per-module way to indicate whether the new API should be used. We'd commit to maintaining zope.component 3.x in parallel with zope.component 4.0. We'd recommend to people *not* to require zope.component 4.0 for a period, or perhaps we'd even not release it for quite a period. The documentation issue is a more severe one. [snip]

I think Jim said once, "we can't ever have backwards incompatibility". Other "serious" platforms like Java or .NET have a similar stance. They deprecate liberally, but never actually break anything.

(Remember java.util.Data and java.util.Calendar?)

No, I don't remember. :)

We may never be able to do that completely, and we may *want* to root out some dodgy bits of code that few people use. But breaking something so fundamental and so commonly used would be criminal, in my book.

Taken into consideration.

...

I think it's important not to do a "big bang" upgrade but instead allow people to upgrade bit by bit. It should be possible to compose an application that mixes code that expects the old semantics with code that expects the new semantics. A bit by bit upgrade I think would ideally be on a per-module basis. I think it's important to make sure we can support such an upgrade *before* we release any of this.

I'm not sure that's going to be possible. As soon as someone does zope.component >= 4.0 in their setup.py, you're screwed.

This implies we don't want to release zope.component 4.0 for a long time yet. Regards, Martijn

On Mon, Nov 30, 2009 at 1:21 PM, Martin Aspeli <optilude+lists@gmail.com> wrote:

Martijn Faassen wrote:

...

This implies we don't want to release zope.component 4.0 for a long time yet.

I think the answer should be "never". :)

I think never is a rather long time. I'd suggest we think about these changes more in the timeline of years. Looking at Python itself or Zope's own former deprecation policies, it seems that policies where we deprecate / warn about API changes in one release and change behavior it one or two releases after that seem to work. They do rely on their being something like a coherent release of some language / framework / toolkit though. And they rely on these releases being made at an interval of at minimum a year or preferably 18 months (as in Python's case). I think that once we get a ZTK 1.0 release out that promises to be maintained for at least three years, we can start working on a ZTK 2.0 which introduces deprecation warnings about the changed behavior and a 3.0 that will change the default. If released at an interval of 18 months like Python, that puts these changes about 3 years into the future with a lot of time in between to adjust. Given such an approach I think we can indeed change core API's in backwards incompatible ways. Python itself does this all the time, look at Exceptions as new-style classes, new language keywords like "with" or the massive amount of changes in Python 3. But if we treat zope.component / zope.interface just as two packages of their own, I'd agree that we don't have any way to provide reasonable backwards compatibility and ensure that some packages won't use these straight away. The whole point of the toolkit is to ensure we have a large number of packages that are compatible and tested with each other. Hanno

Leonardo Rochael Almeida wrote:

I find it rather odd that we're wasting so much time worrying about backward incompatibility when we have a perfect mechanism to introduce backward incompatible changes in a way that allows both flavours to be used by packages in the same application (on a module by module basis just like Martijn would like):

* Use a different package name!

We don't have that option, as we're talking about changing the behavior of calling IFoo. The options are: * changing the signature of calling IFoo in a backwards incompatible way, with various transition strategies. * changing the signature of calling IFoo in an almost backwards compatible way (breaking tuple adaptation) * adding new methods to IFoo. Regards, Martijn

Hanno Schlichting wrote:

On Mon, Nov 30, 2009 at 5:09 PM, Martijn Faassen <faassen@startifact.com> wrote:

...

Leonardo Rochael Almeida wrote:

...

* Use a different package name! We don't have that option, as we're talking about changing the behavior of calling IFoo.

It's very well possible. You create a new distribution called for example "Interface". Now you can write:

import interface import zope.interface

class IFoo(interface.Interface): pass

class IBar(zope.interface.Interface): pass

Depending on what "kind" of interface you have the semantics of calling these are different. Not that I'm proposing to do this, as it leads to a pretty horrible mess, but it's possible.

True. But nitpicking, as all along we're talking about an upgrade to zope.component to allow new semantics.

We are just now getting rid of the old Zope2 Interface package and its usage in Plone land, which has the same kind of incompatibility problem. I expect that old version of interfaces is still going to be with us for a number of years. So experience shows that something so central, even if not used for many important things, has a much much longer lifetime than you'd expect.

Agreed. By taking everything along at the same time in this case I think we avoid this issue somewhat, though. Regards, Martijn

On 11/30/09 13:43 , Hanno Schlichting wrote:

On Mon, Nov 30, 2009 at 1:21 PM, Martin Aspeli<optilude+lists@gmail.com> wrote:

...

Martijn Faassen wrote:

...

This implies we don't want to release zope.component 4.0 for a long time yet.

I think the answer should be "never". :)

I think never is a rather long time. I'd suggest we think about these changes more in the timeline of years.

Looking at Python itself or Zope's own former deprecation policies, it seems that policies where we deprecate / warn about API changes in one release and change behavior it one or two releases after that seem to work. They do rely on their being something like a coherent release of some language / framework / toolkit though. And they rely on these releases being made at an interval of at minimum a year or preferably 18 months (as in Python's case).

I think that once we get a ZTK 1.0 release out that promises to be maintained for at least three years, we can start working on a ZTK 2.0 which introduces deprecation warnings about the changed behavior and a 3.0 that will change the default. If released at an interval of 18 months like Python, that puts these changes about 3 years into the future with a lot of time in between to adjust.

We could also say that we will clean up the API when we move to Python 3. That is a natural breaking point anyway, so it will not any extra pain for users of the ZCA. Wichert.

On 11/30/09 14:45 , Hanno Schlichting wrote:

On Mon, Nov 30, 2009 at 2:39 PM, Wichert Akkerman<wichert@wiggy.net> wrote:

...

We could also say that we will clean up the API when we move to Python 3. That is a natural breaking point anyway, so it will not any extra pain for users of the ZCA.

Except that is precisely what the Python developers have asked everyone not to do. So far the story is that the upgrade to Python 3 can be done largely automatic and a codebase for 2.x and 3.x can be maintained automatically and kept in sync.

In theory. I am convinced that in practice you well end up with code that is un-pretty in both python 2.x and 3.x, and harder to debug. Python 3 also introduces changes that warrant API changes, so not making them could lead to awkward APIs. Personally I will take the liberty to change the API of any of my packages if and when I port them to Python 3. Wichert.

Hey, [Python 3 discussions] I think discussions about Python 3 and changing the API then should be tabled in this thread. We're talking about a timeline where the first steps will take place in the next few months. Realistic small steps, please. (just like we'll need realistic small steps towards Python 3. these are in fact taking place) Regards, Martijn

Martin Aspeli wrote:

Martijn Faassen wrote: [snip]

...

That's why I think it's important to have a:

* a zope.component 3.x that supports both patterns

* a per-module way to indicate whether the new API should be used.

Sorry, I just don't buy it. The *moment* someone requires >= 4.0, you're screwed.

See discussion below.

And per-module flags are ugly and confusing. I'm quite sure most people never import from __future__ in Python, just because it's so hard to see what's going on and so annoying to have to remember to do it.

From future imports are going to be part of our life for a significant period, as we go through Python 2.6 and then presumably, sometime, to Python 3.x.

...

We'd commit to maintaining zope.component 3.x in parallel with zope.component 4.0. We'd recommend to people *not* to require zope.component 4.0 for a period, or perhaps we'd even not release it for quite a period.

This sounds like we're making excuses for making a bad choice. What's the point of releasing a package if we encourage people not to use it? Or not to use it "yet"? When does "yet" end?

We won't be able to control what people do in their code. And it only takes one package to depend on 4.0 for it to all go wrong for anyone using the current, documented, *encouraged* pattern.

So I'm adjusting my story to say we shouldn't release zope.component 4.0 at all. We should first go through zope.component 3.x which gives: * a deprecation error if 'default' is not used explicitly. * a from future mode so that the new semantics can be used on a per-module basis. [snip documentation issue being severe, if not criminal]

...

...

...

I think it's important not to do a "big bang" upgrade but instead allow people to upgrade bit by bit. It should be possible to compose an application that mixes code that expects the old semantics with code that expects the new semantics. A bit by bit upgrade I think would ideally be on a per-module basis. I think it's important to make sure we can support such an upgrade *before* we release any of this. I'm not sure that's going to be possible. As soon as someone does zope.component >= 4.0 in their setup.py, you're screwed. This implies we don't want to release zope.component 4.0 for a long time yet.

I think the answer should be "never". :)

To put this into perspective: we're going to cause a lot of pain for a lot of people for something that is *purely* cosmetic. We're indulging in a whim for "perfect" API design at the expense of people who signed up to our old API.

Sometimes, we need to accept that our past decisions are with us to stay. I think that's a sign of maturity and attention to our customers, rather than weakness.

Tell that to the Python core developers. :) Anyway, I'm a bit more flexible on the issue of backwards compatibility. But the deeper in the stack we are the more careful we should be, indeed, as there are many consumers, directly and indirectly.

There are solutions in these threads which are backwards compatible, or at least backwards compatible in the vast majority of cases. They may not be exactly as pretty, but in my book they are infinitely preferable.

I would much, much rather have none of these improvements and not break all that code. What we have now works. We would just like it to be a little bit prettier and a bit more obvious. Those are laudable goals, but not at any expense.

The most elegant backwards compatible solution would be multi adaptation using a tuple. I think 'name' can probably also be added to the adapter hook without breaking stuff. People adapting tuples will need an explicit way to do so. It's still backwards incompatible, but the impact is less big. In any case, I would like to deprecate non-explicit keywords parameters for 'default' and 'name'. Regards, Martijn

On Mon, Nov 30, 2009 at 08:40, Wolfgang Schnerring <ws@gocept.com> wrote:

Thus, we should not start requiring zope.component 4.0 everywhere immediately (because it's new, great and shiny ;), but rather use 3.9+future when we want to use the new semantics, and only after I don't know, 6 months maybe, start switching over completely.

Six months? :) The last non-alpha release of Plone is 3.3.2, which runs on Zope 2.10.9, which uses Zope 3.3.2, released over two years ago. If we are to break backwards compatibility we need to make a deprecation warning, and let that run until the large body of Plone code has gotten that deprecation warning and been able to move over to either a "future" syntax or some other syntax before we can actually break the backwards compatibility. So if you multiply those 6 month with 5, then maybe that path forward is feasible. If we want to change the API faster, we need a backwards compatible way, and as mentioned, there doesn't seem to be one. On Mon, Nov 30, 2009 at 14:45, Hanno Schlichting <hanno@hannosch.eu> wrote:

On Mon, Nov 30, 2009 at 2:39 PM, Wichert Akkerman <wichert@wiggy.net> wrote:

...

We could also say that we will clean up the API when we move to Python 3. That is a natural breaking point anyway, so it will not any extra pain for users of the ZCA.

Except that is precisely what the Python developers have asked everyone not to do.

This is true. But the fact is that we don't have any choice. The current 2.x syntax simply doesn't work under Python 3. We *must* change the API, and we will do that by moving implements() to @implementor.

So far the story is that the upgrade to Python 3 can be done largely automatic and a codebase for 2.x and 3.x can be maintained automatically and kept in sync.

And this is still true if you write a fixer for it. So that means we must write a fixer that changes IFoo(bla, bleh) to IFoo(blah, default=bleh). Writing fixers are High Magic, but throw tons of testcases on it and some trial and error works. :) So cleaning up the API for Python 3 is fine, IMNSHO. It will be slightly kludgy to implement it though, but doable. On Mon, Nov 30, 2009 at 16:40, Martin Aspeli <optilude+lists@gmail.com> wrote:

That's a nice theory, but experience suggests it'll be a right mess. Is anyone doing this successfully on a project of a comparable size to Zope? Or Plone? It sounds like fantasy to me. Why? Because if the compatibility really was that "mechanical" there would probably be a way to run Python 2 code in Python 3 - and there isn't.

No, of course nobody has done it with a project of comparable size to Zope and Plone. Is there one even? :-) But that wouldn't be a problem I think. Martin von Löwis has made test-ports of both ZODB and Django to Python 3. The problem is that there are tons of developers involved in the Plone community making a lot of popular and well used third-party components, and getting all of them to support both Python 2 and Python 3 at around the same time (I mean within the same year) seems unlikely, and that risks ending up in a catch 22 situation where nobody moves to Python 3 because nobody else has. But that's a different discussion. Although I have no problems with changing the API for Python 3, both that option and the option of making a slow deprecating means that we can't actually break backwards compatibility for a couple of years anyway. What other options are there? -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64

Stephan Richter wrote:

On Friday 27 November 2009, Martijn Faassen wrote:

...

Are people okay with the proposed semantics?

Would people be okay with such an upgrade path? Any better ideas?

Looks good.

Note: We had Thanks Giving over the weekend, so please allow more US people, like Jim, to comment before finalizing the decision.

Good point. We'll give it some more time. Given some feedback about backwards compatibility, I'm leaning to the following adjusted scenario: * allow IFoo((a, b)) for multi adaptation. This breaks tuple adaptation. It's not as pretty as IFoo(a, b), but it's pretty tolerable and it *is* actually symmetric with registration. * deprecate a non-explicit default such as IFoo(a, default), require IFoo(a, default=default) * do the other stuff (name, utility lookups, etc) * this will be a zope.component 3.x release. Or we could even call it 4.0. * we can stick with this for quite a while. * in some years time, see about allowing IFoo(a, b) for multi adaptation. By that time people will have updated their code to use explicit defaults everywhere. * then deprecate IFoo((a, b)) in favor of IFoo(a, b) * we can then allow tuple adaptation again. :) Regards, Martijn

Tres Seaver wrote: [snip]

Do we really have a significant codebase which both needs to adapt tuples *and* uses the interface-calling sugar?

I hope not. That's why I walk all over it breaking backwards compatibility in this plan. We'd need to live with IFoo((a, b)) for a few years as opposed to IFoo(a, b), but if that means we can move forward without breaking a lot of code, I think we should take that hit. Maybe we'll like it enough that we never really need IFoo(a, b) after all. Regards, Martijn

Stephan Richter wrote:

On Monday 30 November 2009, Martijn Faassen wrote:

...

* we can then allow tuple adaptation again. :)

Tuple adaption was also really important to the Twisted guys. We should consult them to see whether they are still using zope.component and whether they are still adapting tuples.

Hm, could I delegate you to contact the right people on this? And whether they are using the adapter hook for it? Regards, Martijn

On Mon, Nov 30, 2009 at 19:16, Shane Hathaway <shane@hathawaymix.org> wrote:

If adding lookup() is a good idea

Possibly, but it sound like you are looking up (a), when in fact you are adapting it. :) Maye IFoo.adapt(a) ? -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64

Shane Hathaway wrote:

Martijn Faassen wrote:

...

Given some feedback about backwards compatibility, I'm leaning to the following adjusted scenario:

* allow IFoo((a, b)) for multi adaptation. This breaks tuple adaptation. It's not as pretty as IFoo(a, b), but it's pretty tolerable and it *is* actually symmetric with registration.

* deprecate a non-explicit default such as IFoo(a, default), require IFoo(a, default=default)

While this short spelling of component lookup is attractive and sensible for Zopistas, I wonder if it ever really was the best idea. When a developer encounters the IFoo() pattern for the first time, what is he/she going to type into a search engine to find out what it means? I can't think of any search phrase that is likely to give a good answer.

When he can't Google, a maintenance developer with no prior ZCA exposure literally sees "IFoo()" (without any args) is going to find the definition for "IFoo" and it will be a class. He will believe that calling it will give him back an instance. This is just consistent with all prior experience he has if he's a Python programmer. Furthermore he'll believe he "owns" the resulting object, because normal classes are always constructors that create a new object. It just can't be obvious to him that "IFoo()" will almost always return some "shared" object (a utility) that isn't an instance of the "class" defined by the IFoo definition. And if a developer is doing maintenance work, he can't afford to track down the docs and become enraptured by the world we create where this isn't the case.

JQuery has a similar issue, but because JQuery is a smaller framework, the JQuery folks simply put that info right near the top of their documentation. I'm not sure we can do that quite as effectively.

For an alternate spelling, consider what happens when component lookup fails: you get a ComponentLookupError. "Lookup" is interesting. It doesn't yet have any meaning in zope.interface, so I think using a method named lookup() would make code more comprehensible. You would use it like this:

...

...

...

IFoo.lookup(a) <SomeAdapter instance at ...> IFoo.lookup(a, b) <SomeMultiAdapter instance at ...> IFoo.lookup(c) Traceback... ComponentLookupError(...) IFoo.lookup(c, default='missing') 'missing' IMyUtility.lookup() <MyUtility instance at ...>

When developers encounter this for the first time, they might type "zope.interface lookup" in a search engine. That phrase has a reasonable chance of hitting good documentation.

What do you think?

+ 1 with the following caveat: I think that method name should probably be "adapt"; "lookup" should maybe be a separate method reserved for passing bare interfaces rather than objects which implement interfaces, e.g:

...

...

IFoo.lookup(IBar) <class FooBarAdapter>

This would be consistent with the nomenclature in the current zope.interface AdapterRegistry API. If it would help to change the resulting error message to "adaptation error" when ".adapt" is called, e.g.:

...

...

IFoo.adapt(c, default='missing') Traceback... AdaptationError(...)

That would be possible too obviously through the magic of subclassing. I think adding methods to the registry object with the same names but slightly different signatures would go hand in hand with such a change: class Components(...): def lookup(self, required, *provided, name=''): ... def adapt(self, required, *provided, name=''): ... sm = getSiteManager() sm.lookup(IFoo, IBar) sm.adapt(IFoo, bar) - C

Am 30.11.2009, 20:24 Uhr, schrieb Gary Poster <gary.poster@gmail.com>:

1) I very much like the idea of some helpers hanging around. However, my current belief is that the factory "methods" ought to be callable objects that allow introspection of the underlying registry. That's where the "lookup" style behavior belongs, IMO, as well as other helpers. See below for examples.

2) As argued before, I think that "adapt" is an ok name for a single object, but becomes a bad name once you have "multiadapters" in the mix. I would prefer one of the options Matthias Lehmann proposed ("new" for instance).

I have no great problem with multiadapters as long as the analogy is clear enough - "this adapter takes two sources..."

3) I also think that "utility" is a bad name. Is "singleton" two letters too long? If it is, I mind "utility" less than I mind "adapter".

I don't understand this. For me a singletons is (sic) a highly specific programming term whereas adapters and utilities, especially in the way we refer to them, are not so domain specific.

IFoo.new(a, b) # finds and returns result of call IFoo.new.lookup(IA, IB) # finds and returns callable IFoo.new.find(IA, IB) # get all registration information IFoo.new.find_stack(IA, IB) # get an iterable of the stack all registration information for each registration for those two interfaces IFoo.singleton() # finds and returns item IFoo.singleton(name='baz') # finds and returns item IFoo.singleton.lookup(name='baz') # same result in this case IFoo.singleton.find(name='baz') # get all registration information

Interestingly this is starting to look too verbose and java like to me but I'm also not happy with the use of "new" or "singleton". "find" might be an idea if it could use introspection to gives clues as to what I might actually want to do with my IFoo implementers. Can you give some sample responses?

Side, but related point: I wonder if there is value in the ability to spell

Could someone please point me in the direction of the definition of this use of spell? Is it short for "spell it out"? Charlie -- Charlie Clark Managing Director Clark Consulting & Research German Office Helmholtzstr. 20 Düsseldorf D- 40215 Tel: +49-211-600-3657 Mobile: +49-178-782-6226

On Nov 30, 2009, at 4:40 PM, Gary Poster wrote:

Put yet another way, how are 99+% of our "utility" usages not singletons?

Therein lies the problem. Singletons are singletons in 100% of cases. Since utilities are not singletons in 100% of cases they are not singletons by definition.

If that's the case, what's the value of having to explain what a utility is?

There is nothing to explain. Utility is something useful that helps you accomplish a task. Which task? Well, the one you just looked a utility for. :-) Zvezdan

On Mon, Nov 30, 2009 at 5:14 PM, Lennart Regebro <regebro@gmail.com> wrote:

True. For me utilities are tools. Like CMFs portal_whatever. But in Zope3 even small stupid singleton objects are utilities in some cases, and that is confusing for a beginner.

I wonder how many typical Python programmers know the term "singleton". Though it's not unusual for there to be exactly one instance of a class in a process, it's pretty unusual to think about that as a valuable aspect of a class. Which for the traditional definition of singleton, it very much is. -1 for calling utilities singletons, since that has nothing to do with their usage. +1 for calling them utilities, since that has everything to do with how they're used. -Fred -- Fred L. Drake, Jr. <fdrake at gmail.com> "Chaos is the score upon which reality is written." --Henry Miller

On Tue, Dec 1, 2009 at 01:16, Gary Poster <gary.poster@gmail.com> wrote:

I don't love "singleton."  I think it is better than "utility."  I agree at least that it probably isn't better enough to introduce confusion.  I'd be more insistent on "singleton", or finding a better term than either of them, if this were a fresh API.

I wonder if the best isn't to make the documentation clearly say "Utilties are singleton components that can be looked up per interface". -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64

On Nov 30, 2009, at 5:14 PM, Lennart Regebro wrote:

On Mon, Nov 30, 2009 at 22:40, Gary Poster <gary.poster@gmail.com> wrote:

...

Then to the multiadapter concern I raised, all my real-world examples of adapters are to adapt one object so it can be used in a certain way (to integrate with another kind of object). Power adapters, for instance, adapt a plug (required interface) so it can plugged in to the wall (output interface). Is there a common real-world example of this for "multiadapters"?

Yup. http://www.amazon.co.uk/Scart-Adapter-Switchable-Plug-Socket/dp/B00077DC6A

Audio + Video in: SCART out. :)

heh. And Shane's example was more commonplace. I still think this is unusual, or in Shane's example, not something that people think of as a multiadapter. But as said, to Fred, point partly taken. :-) [snip utility/singleton]

...

That said, and to repeat, I mind "adapter" more than "utility."

But adapter is really what it is. OK, Multiadapters are evidently complicated... But is it really so complicated that we should throw away the commonly accepted GoF for what clearly are adapters? How is it less confusing to call IFoo.instance(x,y) than IFoo.adapt(x,y) or even IFoo(x,y)?

I am very much in favor of IFoo(x, y). That makes me very happy. It looks mostly like you are instantiating a class, except that it looks a bit funny: in my view, it is a reasonably good leaky abstraction for what is going on. People also like the compactness of the spelling, in my discussions. They also remember it very well, even over long periods of not using the API. This is a big deal. Backwards compatibility is the problem. I need to go have a life. :-) Talk to you all tomorrow. Gary

Martijn Faassen wrote:

Hi there,

Thanks for doing the summarise-and-move-it-on job so well, Martijn. It's really important.

I'd like to summarize the options I've seen appear in the discussion so far.

We have the following options:

1) introduce a new method, such as "instance()" or "lookup()" on instance. It unifies utilities with adapters. We can make it do whatever we want without worrying about backwards compatibility.

I find the idea of trying to conceptually reconcile adapters and utilities pretty weird. They are quite different concepts. Searching for one over-arching metaphor will probably land us in an even more conceptual "applies-to-everything" situation. I've spent the past few years explaining adapters and utilities to people as two fundamental types of components. I'd really rather not try to bring it "up" one level to explain something more generic (and thus less specific/example-friendly). I could be convinced otherwise, I'm just saying that I don't see why trying to "unify" these two things is important.

2) introduce several new methods that distinguish between utility and adapter lookup. We can make them do whatever we want without worrying about backwards compatibility.

3) call the interface, which unifies adapter and utility lookups.

If all you mean by "unify" is "more consistent API", then +1. I saw other things in the thread related to trying to treat the two as different types of the same fundamental thing, which I think we want to avoid.

Use tuples for multi adaptation. We think could make this work without *too* much backwards compatibility issues (pending research on how prevalent tuple adaptation really is). In the long term we can even map out a deprecation strategy that can smoothly migrate us to a "multi argument" approach.

+1, though I think the multi-argument thing is still going to be wrong in the future, and won't bring enough benefit for the pain it will cause. Even if by then there's no BBB to be concerned about, it's still a change, and it still breaks documentation.

4) call the interface, which unifies adapter and utility lookups. Use multiple arguments for multi adaptation. The backwards compatibility obstacles are largest here as we already have the "default" argument. We'd need to introduce multiple "modes" to selectively upgrade.

-1 (aka "minus a lot"), as I've said before.

I'm in favor of calling the interface. I'm also in favor of unifying adapter and utility lookup.

On the back end, I'm also in favor of allowing utility creation by factory (or "null adaptation")

Can you summarise what you mean by this? The thread is so long...

and allowing instance lookup for instances ("contextual utility lookup" or "adaptation to an instance").

My brain hurts... examples?

I think four ways to retrieve an object of the right interface (combining factory/registered instance and lookup globally/lookup for an instance) is a good argument *against* distinguishing between creation strategies or "connection to adapted object or not" in the API.

I'm afraid you've lost me. Four ways sounds bad, though. ;-)

If I look up a utility I wouldn't want to care whether it happened to be a previously registered instance or a factory created one. If I look up an adapter I wouldn't care whether it happened to be a previously registered instance either. In fact, returning a previously registered instance can be very well implemented using a factory. (In fact, this suggests to me we should actually explore implementing instance registration in terms of special factories.)

Heh, I used the "shared adapter implementation" as an example of custom factories in http://plone.org/products/dexterity/documentation/manual/five.grok/core-comp... Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book

Martijn Faassen wrote:

First a statement about the goal of this discussion. The goal of this discussion is to convince people to unify the lookup API. I wouldn't want to make lookup API improvements depend on improvements to zope.component inspired by the discussion below. I'm in favor of exploring that in a separate project, however.

+1

* abstract factory called on an object (adaptation)

In: the requested interface, one ore more instances

Process: look up factory. Call factory with input instances.

Out: a new instance that provides requested interface

* abstract instance retrieval (utility lookup)

In: the requested interface

Process: look up instance.

Out: a previously registered instance that provides requested interface.

* abstract factory not called on an object ("utility factory", "null-adaptation")

In: the requested interface

Process: look up factory. Call factory.

Out: a new instance that provides the requested interface

* abstract instance retrieval for an object ("utility associated with an instance", "adapting to an existing instance")

In: the requested interface, one or more instances.

Process: look up instance with input instances as "context".

Out: a previously registered instance that provides the requested interface

The two latter patterns aren't fully supported. All of them could in principle be emulated using the "look up factory/call factory" pattern.

I think the last pattern can be implemented with a custom adapter factory. You'll need custom "retrieval" code anyway, so there's probably not much more room for abstraction.

There are two perspectives on distinguishing adapters from utilities. One perspective is:

* newly created instance (adapter) versus previously registered instance (utility).

This perspective informs the "singleton" discussion. I'd say a better way to distinguish adapters from utilities (should we wish to do so) is:

* arguments into the factory (adapter) versus no arguments into the factory (utility)

That's inspired by the notion that adapters tend to have some form of abstract "connection" to what they adapt, while utilities do not.

This is certainly true, but does that really inform how we use these things? To me, I use unnamed utilities for: * singletons (as in, one implementation) * strategies (as in, I look up some configuration for an algorithm by asking the CA for an object providing a particular interface) I use named utilities for: * registries of homogeneous objects (to avoid implementing my own registry) I use adapters for: * the adapter pattern (my code expects an IFoo; the client code can write an adapter factory to get me one from whatever context it passes in) * customisation/specialisation (I look up an adapter to fulfil some policy, allowing it to be overridden with more specific adapters) And a few other variations (such as the "retrieve object" use case, but that's very rare and has largely been an implementation detail). I don't think of these in terms of "how many arguments" the component needs. I think of them in terms of various software patterns. And to me, an adapter provides the ability to work with different "aspects" of an object (adapting it to different interfaces) whilst utilities provides a way to get the "current" implementation of a particular service or policy. I'm very much for consistent APIs, and if we can simplify the registry implementation with more shared code, that'd be great. I'm just really worried that we try to "simplify" by replacing two concepts we've spent ages teaching people, with a single concept that is even more abstract and used in even more different ways for even more purposes. You start having to use lots of words to disambiguate ("named multi argument factory component" == a view). I don't think we win there. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book

Martijn Faassen wrote:

Hi there,

I'd like to summarize the options I've seen appear in the discussion so far.

We have the following options:

1) introduce a new method, such as "instance()" or "lookup()" on instance. It unifies utilities with adapters. We can make it do whatever we want without worrying about backwards compatibility.

2) introduce several new methods that distinguish between utility and adapter lookup. We can make them do whatever we want without worrying about backwards compatibility.

3) call the interface, which unifies adapter and utility lookups. Use tuples for multi adaptation. We think could make this work without *too* much backwards compatibility issues (pending research on how prevalent tuple adaptation really is). In the long term we can even map out a deprecation strategy that can smoothly migrate us to a "multi argument" approach.

4) call the interface, which unifies adapter and utility lookups. Use multiple arguments for multi adaptation. The backwards compatibility obstacles are largest here as we already have the "default" argument. We'd need to introduce multiple "modes" to selectively upgrade.

I'm in favor of calling the interface. I'm also in favor of unifying adapter and utility lookup.

On the back end, I'm also in favor of allowing utility creation by factory (or "null adaptation") and allowing instance lookup for instances ("contextual utility lookup" or "adaptation to an instance"). I think four ways to retrieve an object of the right interface (combining factory/registered instance and lookup globally/lookup for an instance) is a good argument *against* distinguishing between creation strategies or "connection to adapted object or not" in the API.

If I look up a utility I wouldn't want to care whether it happened to be a previously registered instance or a factory created one. If I look up an adapter I wouldn't care whether it happened to be a previously registered instance either. In fact, returning a previously registered instance can be very well implemented using a factory. (In fact, this suggests to me we should actually explore implementing instance registration in terms of special factories.)

I am also in favor of unifying adapter and utility lookup. Or at least creating a more normalized API. On the syntax of the change: I am more or less somewhere between -0 and +0 on the idea of presenting a unified API as methods of interfaces (call or non-call). While I don't think this idea is the worst idea in the world and it might be better than the current global API, I don't think we should *only* do this; similar changes should be made to the registry itself. Code that uses the global API needs to do a registry lookup for each usage, so it's slower than code which doesn't. If interface methods become available to unify adapter and utility lookup, the same sorts of methods should be added to the registry itself for performance-sensitive code. I also believe that code that locates a registry explicitly and calls lookup methods on it is easier to understand for the maintenance programmer than is an equivalent global API. On the semantics of the change: Personally I think that it's a fantasy to believe that the difference between an object created via a factory on-demand and an object simply returned should *never* matter to a caller. You may not want the caller to need to care, and it may be inconvenient to take into account circumstances where the caller needs to care. But because this is Python, the caller still often does need to care. The only circumstance where a caller would *never* need to care would be when the object returned could be guaranteed to *never ever* have any methods called on it that weren't part of the interface defined by the interface being looked up. If we want this, we should probably just be using Java. I am extremely uncomfortable with any situation where: class IFoo(Interface): pass IFoo() ... does not result in any factory invocation. I hope it's obvious why this is the case. The current global API has a modicum "kindness" for maintenance developers because we *haven't* unified adapter and utility lookup in such a way. They stand a shot at understanding that the result of getUtility usually has "lifetime A" and the result of getAdapter/getMultiAdapter has "lifetime B". If we take out this safety belt, I'd lean towards more explicitness rather than less: let the caller call the result of the lookup. - C

Chris McDonough wrote:

On the semantics of the change:

Personally I think that it's a fantasy to believe that the difference between an object created via a factory on-demand and an object simply returned should *never* matter to a caller. You may not want the caller to need to care, and it may be inconvenient to take into account circumstances where the caller needs to care. But because this is Python, the caller still often does need to care.

The only circumstance where a caller would *never* need to care would be when the object returned could be guaranteed to *never ever* have any methods called on it that weren't part of the interface defined by the interface being looked up. If we want this, we should probably just be using Java.

I'll retract this. If you want to create a world where callers never need to care about the lifetime of any object returned by a component lookup, you could also return a proxy wrapper around the returned object that only allows for the invocation of the methods defined in the interface looked up. Any other access or invocation would raise an exception. This would make this worldview 100% consistent, completely documentable, and would cause no confusion during use. It would thus remove any logical argument I have against it. I wouldn't be eager to use such a system, but it's totally consistent. - C

Chris McDonough wrote:

Personally I think that it's a fantasy to believe that the difference between an object created via a factory on-demand and an object simply returned should *never* matter to a caller. You may not want the caller to need to care, and it may be inconvenient to take into account circumstances where the caller needs to care. But because this is Python, the caller still often does need to care.

The only circumstance where a caller would *never* need to care would be when the object returned could be guaranteed to *never ever* have any methods called on it that weren't part of the interface defined by the interface being looked up. If we want this, we should probably just be using Java.

I think it's okay for us to say: if you call a method on something that you retrieved by interface that isn't actually specified to be on that interface, you're on your own. It may work, but it's not specified to work by the system you're using. I'm not sure why lifetime issues are considered so important here. Of course they sometimes matter. But we use ORMs all the time. We create traversal hierarchies on the fly all the time. What's different in this case? If I want to index an object, do I care whether the catalog was just created on the fly and it's talking to a relational database? I think you're arguing that because abstractions are sometimes leaky (lifetime issues) we shouldn't have this abstraction at all. Why is the leakiness so important here? [snip]

The current global API has a modicum "kindness" for maintenance developers because we *haven't* unified adapter and utility lookup in such a way. They stand a shot at understanding that the result of getUtility usually has "lifetime A" and the result of getAdapter/getMultiAdapter has "lifetime B". If we take out this safety belt, I'd lean towards more explicitness rather than less: let the caller call the result of the lookup.

I think there are two centers of attraction for unification: * everything is a factory. If I want to look up an instance, I look up a factory that returns me the same instance all the time. I.e. IFoo() and IFoo(a) * everything is an instance. If I want to call a factory, I look up an instance that is a factory, and call it. I.e. IFoo.lookup() and IFoo.lookup(a)(a) zope.component presents a compromise between those positions. The advantage I see of unifying towards factories is a less verbose API for common operations than an explicit API would have. If you take the utilities come from a factory approach, this would suggest an API like this (on Interface): def __call__(self, arg=()): if not isinstance(arg, tuple): arg = (arg,) return self.factory(arg)(*arg) # low level def factory(self, arg=()): return registry.lookup(from_=arg, to=self) The drawback is that while you *can* register utilities as non-factories and look them up, you'd do it through 'factory', as the real way to look up utilities would be to call a factory. If you take the utilities are registered instances, that would suggest an API like this: # convenience def __call__(self, arg=()): if not isinstance(arg, tuple): arg = (arg,) return self.lookup(arg)(*arg) def lookup(self, arg=()): return registry.lookup(from_=arg, to=self) You'd register utilities as instances directly and look them up using lookup(). The convenience method still works for adapters. The drawback is that IFoo() would result in IFoo.lookup()(), and calling the utility would make no sense. The implementations are the same, but the semantics are quite different. :) Regards, Martijn

Chris McDonough wrote:

I am more or less somewhere between -0 and +0

That is a high degree of precision. Maybe we need to start thinking of our voting system as a Decimal instead of an int? Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book

Am 01.12.2009, 17:08 Uhr, schrieb Lennart Regebro <regebro@gmail.com>:

I'm +1 for 5.

"wot he said" x 10 :-) The video + audio to scart example struck me last night and providing expressive(?) examples of this to match so that one is less easily caught by tuples as arguments is a doddle. cables = (audio, video) connector = IScart.adapt(cables, "HDTV") Assuming I haven't got totally the wrong end of the stick this shouldn't confuse anyone. Now, do we have any similar simple but expressive analogy for utilities? Soap dispensers? ISoapDispenser.utility().dispense() # agained, spelled out for clarity seeing as that's all the utility does. I hope my analogies aren't too far off the mark. Helps me a lot not to think about what's really going on. Charlie -- Charlie Clark Managing Director Clark Consulting & Research German Office Helmholtzstr. 20 Düsseldorf D- 40215 Tel: +49-211-600-3657 Mobile: +49-178-782-6226

Gary Poster wrote:

You are leaving out the variants of 3 and 4 that allow calling the interface to support multiadaptation, but do not unify utilities.

True, my mistake. Lennart pointed that out too just now.

My impression is that I am not the only one who is not pleased with the proposed unification of utilities and adaptation.

I've tried to analyze why that might be so elsewhere. There are three perspectives that I can see: * everything registered is really a factory. Returning a utility just requires registering a special factory. * everything registered is really an instance. Calling an adapter is just looking up an instance and then calling it. * we have two kinds of things registered, instances and factories. zope.component does that today.

My impression is that we are nearing consensus on the variation of 3 that does not include utilities.

I could live with that, as long as we had a way to look up utilities on the interface too. Regards, Martijn

Gary Poster wrote:

Then to the multiadapter concern I raised, all my real-world examples of adapters are to adapt one object so it can be used in a certain way (to integrate with another kind of object). Power adapters, for instance, adapt a plug (required interface) so it can plugged in to the wall (output interface). Is there a common real-world example of this for "multiadapters"?

I have a Roku player (great device, BTW). It streams video from Netflix and other sources. It takes two inputs (network and power) and produces one output (a video signal). I could call the device a multi-adapter, but the power input is so simple and reliable that I forget about it. Most of the time I think of the Roku player as a simple adapter from Internet packets to a video signal, but electrically, it's definitely a multi-adapter. Note that the network signal for a Roku player varies wildly, while the power is either ~110VAC or ~220VAC. Multi-adaptation works best when it has similar characteristics, I think. It's safe to allow one of the inputs to multi-adaptation to vary a lot, but to keep developers sane, the rest of the inputs should be more predictable. I think getMultiAdapter((context, request)) is OK because most web sites have only one or two request types.

Turned around, people know the term "singleton" and they do not know the terms "adapters" and "utilities". "singletons" describe the huge majority of how we use these things. It's something less to explain. Making comprehension quicker is very valuable to me.

Do you intend to change the API names in zope.component, then? For example, getUtility -> getSingleton? That might be possible, but no one has suggested it before (AFAIK), and I think it's implied by your suggestion.

``IFoo.new(a, b)`` is equivalent to getMultiAdapter((a, b), IFoo)

Using "new" for a name could be a problem for Jython and IronPython users, since new is a keyword in other languages. Shane

Gary Poster wrote:

Then to the multiadapter concern I raised, all my real-world examples of adapters are to adapt one object so it can be used in a certain way (to integrate with another kind of object). Power adapters, for instance, adapt a plug (required interface) so it can plugged in to the wall (output interface). Is there a common real-world example of this for "multiadapters"?

It's a good question. Plugging a power cord in on both ends? :) Slightly more seriously, a lego brick connecting to multiple other bricks? But that isn't a good enough answer to that question.

...

...

3) I also think that "utility" is a bad name. Is "singleton" two letters too long? If it is, I mind "utility" less than I mind "adapter". I don't understand this. For me a singletons is (sic) a highly specific programming term whereas adapters and utilities, especially in the way we refer to them, are not so domain specific.

Turned around, people know the term "singleton" and they do not know the terms "adapters" and "utilities". "singletons" describe the huge majority of how we use these things. It's something less to explain. Making comprehension quicker is very valuable to me.

I don't like the word singleton very much either. Singleton in the Design Patterns book has a very particular implementation that is criticized by a lot of developers and in particular that particular pattern is very uncommon in the Python world (people just use globals). I think introducing the term would pull in all that baggage to a newcomer. Just type in 'singleton' in Google and you'll get the wikipedia definition: In software engineering, the singleton pattern is a design pattern that is used to restrict instantiation of a class to one object. Utility classes are *not* restricted to a single instantiation. Now we can argue successfully that's an extension of the singleton principle, but then we've lost a lot of people already who thought they knew (or went to lookup) what the word "singleton" means. Regards, Martijn

Martin Aspeli wrote:

For the record, I normally use the singleton analogy to explain unnamed global utilities. Perhaps that's bad, though I find it works pretty well. It'd probably be more accurate to use the terms you did, an "extension of the singleton principle", but as you say, it just adds more complexity.

To me, the Singleton pattern says, "each time you ask for this, you get the same object". That's a little bit different from "this class can only be instantiated once". I think the important part of the design pattern is the shared instance (e.g. to conserve resources or to implement some kind of shared counting/tracking), not the restrictions on instantiation.

I'm certainly -1 on using the term in the ZCA. I think changing our nomenclature would be terrible for the same reasons I think changing (as opposed to extending/improving) our API would be terrible. Utilities are in one way a more specific concept (due to the lookup semantics) and in another way a more generic concept (since named utilities can be used to implement a registry of homogenous objects).

To me the fact that an object "is" a singleton or a factory is orthogonal to the registry stuff. Why can't utilities be factories too that simply return themselves when being called? Then being a singleton or not would be in the responsibility of the registered object (class, factory, singleton) and the ZCA would not need to know. What am I missing (except bbb)?

Martijn Faassen wrote:

Joachim König wrote: [snip]

...

To me the fact that an object "is" a singleton or a factory is orthogonal to the registry stuff.

Why can't utilities be factories too that simply return themselves when being called? Then being a singleton or not would be in the responsibility of the registered object (class, factory, singleton) and the ZCA would not need to know.

What am I missing (except bbb)?

I don't think you're missing anything. It's just that others have a different perspective on this than you and me.

Clearly, it could. But that's not the way we went. Changing it now would be really damaging, and I'm not sure what would be gained. I can imagine use cases where getting a new instance each time would be useful. I can't say I've ever actually needed it, though. If it's something that can be done without breaking existing code or requiring the rewriting of history on patterns we've encouraged and documented to date, it'd certainly be an interesting option to explore. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book

Joachim König wrote:

Martin Aspeli wrote:

...

Clearly, it could. But that's not the way we went. Changing it now would be really damaging, and I'm not sure what would be gained.

I can imagine use cases where getting a new instance each time would be useful. But that is under the full controll of the __call__ of the utility, it could return whatever it wants, as long as what it returns implements the requested interface of course ( imagine a pool of utilities implementing the interface, some being "busy")

I wouldn't want to force everyone to implement def __call__(self): return self Since this is the most common (and current) use case. And ignoring BBB is not an option. :) Nor would I want some separation of factory and object where everyone had to implement both.

If you'd like to check (as a user of the ZCA) if you got a singleton for a utility, then compare the lookup() against the utility returned, e.g. __call__ returned self.

You wouldn't.

The distinction between utility and adapter only burdens the ZCA with no gain. The only reason for ZCA to know about it today is to decide if to return the registered object or to call it. And a lot of discussion is necessary to explain the difference.

I disagree. I think it may burden the internal implementation of the ZCA. I don't think it that's the correct perspective, though. I think that logically, these are two different concepts that meet two different sets of use cases. I think there'd be *more* documentation required to explain how one super-general concept stretches to a dozen different use cases, than to explain how two concepts stretch to half a dozen each in two broad categories. See my reply to Martijn for more detail. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book

Fred Drake wrote:

On Tue, Dec 1, 2009 at 11:34 AM, Martin Aspeli <optilude+lists@gmail.com> wrote:

...

I think that logically, these are two different concepts that meet two different sets of use cases.

Agreed here. This is essential to this discussion.

I've been quite surprised that there are so many who argue to unify these ideas.

-Fred

Agreed here as well. I think that the different intentions of 1) lookup by interface in registry and 2) adaptation will be easier to understand if the code patterns are different as well. Because I doubt tuple adaptation is that frequent, I am for

5) Call the interface for adaption, and something else for utility lookup, with tuples for multi-adaptation.

where the something else is 'Interface.getUtility()'. This way we can - make the interface more prominent when doing lookups (and have some sort of symmetry between lookup and adaptation) - and simultaneously keep the semantic difference between adaptation and lookup. -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be

On Nov 30, 2009, at 2:24 PM, Gary Poster wrote:

3) I also think that "utility" is a bad name. Is "singleton" two letters too long?

Yes and not because "singleton" is longer. It just a bad name. :-)

On Nov 30, 2009, at 4:13 PM, Zvezdan Petkovic wrote:

On Nov 30, 2009, at 4:05 PM, Zvezdan Petkovic wrote:

...

On Nov 30, 2009, at 2:24 PM, Gary Poster wrote:

...

3) I also think that "utility" is a bad name. Is "singleton" two letters too long?

Yes and not because "singleton" is longer. It just a bad name. :-)

To clarify because of

1. the typo above (should be "It's just ..."); 2. the preposition "it" used.

I meant: "Singleton" is a bad name.

I've given my reasons (the most recent attempt was to Charlie Clark). You give yours. :-) Gary

Chris McDonough wrote:

Furthermore he'll believe he "owns" the resulting object, because normal classes are always constructors that create a new object.

Except when they don't. Apart from cases like short strings and small integers where Python itself doesn't create objects more than once, you can always implement classes that define __new__ or use metaclasses in such a way that you cannot be sure that (or whether) calling them under given circumstances will create new objects. To be honest, I just don't see why this whole singleton business shouldn't be orthogonal to the concepts of the component architecture. -- Thomas

On Dec 2, 2009, at 8:33 AM, Fred Drake wrote:

On Wed, Dec 2, 2009 at 2:21 AM, Thomas Lotze <tl@gocept.com> wrote:

...

To be honest, I just don't see why this whole singleton business shouldn't be orthogonal to the concepts of the component architecture.

Well said. If an application cares about singleton creation or ownership of factory-returned objects, it can describe those requirements using interfaces.

You are arguing for the unification of utilities and adapters? Gary

On Wed, Dec 2, 2009 at 8:42 AM, Gary Poster <gary.poster@gmail.com> wrote:

You are arguing for the unification of utilities and adapters?

No. I'm arguing not to conflate utilities with the singleton pattern or adaptation with ownership of the resulting adaptation. -Fred -- Fred L. Drake, Jr. <fdrake at gmail.com> "Chaos is the score upon which reality is written." --Henry Miller

Gary Poster wrote:

Without this distinction, AFAICT either you want to conflate the ideas, or you have a concept of the differences between the two that is more esoteric than I think is useful. I get the impression that it is on the second point of those that we disagree.

Right, I understand the motivation behind your arguments, and I do have a different opinion. OTOH, it's probably helpful for the discussion to have spelled this disagreement out. -- Thomas

On Wednesday 02 December 2009, Lennart Regebro wrote:

When there is such a clear and distinct conceptual difference between adapters and utilities, why would we try to murk that distinction by pretending that non adapters are a special case of adapters, when it's obvious that that's exactly what they are not. "Null-adaptation" is a contradiction in terms. It requires adapters that does not adapt. It can only lead to complete confusion.

I concur. There is no point in merging the concepts of adapters and utilities, since they are fundamentally different. Regards, Stephan -- Entrepreneur and Software Geek Google me. "Zope Stephan Richter"

Martijn Faassen wrote:

Stephan Richter wrote:

...

On Friday 27 November 2009, Martijn Faassen wrote:

...

Are people okay with the proposed semantics?

Would people be okay with such an upgrade path? Any better ideas? Looks good.

Note: We had Thanks Giving over the weekend, so please allow more US people, like Jim, to comment before finalizing the decision.

Good point. We'll give it some more time.

Given some feedback about backwards compatibility, I'm leaning to the following adjusted scenario:

* allow IFoo((a, b)) for multi adaptation. This breaks tuple adaptation. It's not as pretty as IFoo(a, b), but it's pretty tolerable and it *is* actually symmetric with registration.

+1

* deprecate a non-explicit default such as IFoo(a, default), require IFoo(a, default=default)

+0

* do the other stuff (name, utility lookups, etc)

+1

* this will be a zope.component 3.x release. Or we could even call it 4.0.

I'd say 4.0 is more appropriate. This gives us some room to have further 3.x releases in-between/afterwards.

* we can stick with this for quite a while.

* in some years time, see about allowing IFoo(a, b) for multi adaptation. By that time people will have updated their code to use explicit defaults everywhere.

+0

* then deprecate IFoo((a, b)) in favor of IFoo(a, b)

* we can then allow tuple adaptation again. :)

+0 This seems like a more reasonable compromise to me. Cheers, Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Gary Poster wrote:

On Nov 27, 2009, at 6:32 AM, Martijn Faassen wrote:

...

Utility lookups versus adapter lookups --------------------------------------

There was some discussion on whether utility lookups are really something fundamentally different than adaptation as adaptation *creates* a new instance while utility lookup uses a registered instance. I think the essential part here is however: "give me an instance that implements IFoo", and utility lookup fits there. We could even envision a way to create utilities that *does* instantiate them on the fly - it shouldn't affect the semantics for the user of the utility.

As above, I disagree.

The root of the disagreement here is that you seem to want the *caller* to care about something which is important only to the person who *registers* the thing being looked up. From the caller's perspective, the call site needs an object implementing IFoo, looked up using some number N of context arguments, where N could be 0 (no context required to find the object). The fact that, under the hood, an adapter lookup happens to call a factory, passing the context args, is not relevant *to the caller*. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAksT7n8ACgkQ+gerLs4ltQ6vZwCfTT8aWbm4WO7Ba6nQiNPohM3Y QWsAnRUtVRFFQlDRbpnyRao0NZA/mjo3 =VfyQ -----END PGP SIGNATURE-----

Tres Seaver wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Gary Poster wrote:

...

On Nov 27, 2009, at 6:32 AM, Martijn Faassen wrote:

...

...

Utility lookups versus adapter lookups --------------------------------------

There was some discussion on whether utility lookups are really something fundamentally different than adaptation as adaptation *creates* a new instance while utility lookup uses a registered instance. I think the essential part here is however: "give me an instance that implements IFoo", and utility lookup fits there. We could even envision a way to create utilities that *does* instantiate them on the fly - it shouldn't affect the semantics for the user of the utility. As above, I disagree.

The root of the disagreement here is that you seem to want the *caller* to care about something which is important only to the person who *registers* the thing being looked up. From the caller's perspective, the call site needs an object implementing IFoo, looked up using some number N of context arguments, where N could be 0 (no context required to find the object). The fact that, under the hood, an adapter lookup happens to call a factory, passing the context args, is not relevant *to the caller*.

I understand that the idea explained above is conceptually integral to a lot of people, and basically unquestionable. But as devil's advocate sort of thing can we put this traditional worldview aside for a minute, and just sort of take this from ground zero? In "normal Python", callers often do need to understand whether the function they're calling is a factory which constructs a new object, or a function which returns a "global", because the caller needs to know what the impact of mutating the result is. We call non-factories utilities and we call factories adapters. So the caller *already* needs to make a distinction between the two. - C

Chris McDonough wrote:

Tres Seaver wrote: [snip]

...

The root of the disagreement here is that you seem to want the *caller* to care about something which is important only to the person who *registers* the thing being looked up. From the caller's perspective, the call site needs an object implementing IFoo, looked up using some number N of context arguments, where N could be 0 (no context required to find the object). The fact that, under the hood, an adapter lookup happens to call a factory, passing the context args, is not relevant *to the caller*.

I understand that the idea explained above is conceptually integral to a lot of people, and basically unquestionable. But as devil's advocate sort of thing can we put this traditional worldview aside for a minute, and just sort of take this from ground zero?

In "normal Python", callers often do need to understand whether the function they're calling is a factory which constructs a new object, or a function which returns a "global", because the caller needs to know what the impact of mutating the result is.

I think this more often has to do with knowing whether an object should be treated as if it's immutable or not. Often you construct objects and when they're done you only consult them and don't manipulate them anymore. The "traditional world view" works best for objects treated as immutable - you can apply the flyweight pattern (caching) more easily for instance.

We call non-factories utilities and we call factories adapters. So the caller *already* needs to make a distinction between the two.

That's a good point. Let me generalize a bit here below. Adaptation in ZCA is a combination of Design Pattern's abstract factory pattern and the adapter pattern. A utility is something you get back by calling a similar abstract function, but you get back an instance that was already registered previously. Marius and Gary discussed introducing more symmetry. Here is the symmetrical picture as I see it: * abstract factory called on an object (adaptation) In: one ore more instances providing some interfaces, the requested interface Out: a new instance created by a factory that provides the requested interface * abstract instance retrieval (utility lookup) In: the requested interface Out: a previously registered instance that provides the requested interface. * abstract factory not called on an object ("utility factory", "null-adaptation") In: the requested interface Out: a new instance created by a factory that provides the requested interface * abstract instance retrieval for an object ("utility associated with an instance", "adapting to an existing instance") In: one or more instances providing some interfaces, the requested interface Out: a previously registered instance for that object that provides the requested interface There is also the issue of connections: * an adapter typically has a connection to the adapted object. It's not required, however. This is possible because it gets instantiated with the adapted objects as arguments. * a utility never has a connection. That's because it already got instantiated long before the lookup takes place. Whether you see the existence of a connection as essential probably influences whether you prefer the term "adapter" or "utility" in the two latter cases. Here I've looked at the inside of adapters and utilities, and I've also looked at how these things get created. Now back to the "traditional perspective", which I think while not incontrovertible is still extremely valuable. I like the pattern where the caller shouldn't need to know *how* the returned object is created. This suggests unifying utility and adapter lookup. So: IFoo.instance() IFoo.instance(a) IFoo.instance(a, b) "Give me an instance of IFoo (given these objects)". Underneath it could go instantiate IFoo right then and there, possibly passing the arguments to the factory, or it could retrieve an existing IFoo from some registry somewhere. In design patterns terms, a factory could *always* be called. It's just that sometimes it turns around and returns a previously registered instance. (if we are going for a method on an interface, it's clear that "instance" is better than "new", as new implies a new instance while "instance" doesn't imply this. Given it's just one method I'd still be inclined to just call the interface directly) Regards, Martijn

Thomas Lotze wrote:

Martijn Faassen wrote:

...

* a utility never has a connection. That's because it already got instantiated long before the lookup takes place.

Isn't it the other way around: A utility never has a connection to any adapted object, and that's *why we can* instantiate it long before the lookup takes place.

I think the difference between these two perspectives may have to do with why some people in this discussion confuse (as I see it) the concepts of instance vs. factory registration and adapter vs. utility lookup.

I'm not confused. I understand this worldview. I'm just arguing that this worldview is less understandable for new users and maintenance programmers than it would be to leave them distinct (or to "unify" them by providing an API like "lookup" which never calls the result of an adaptation). The conceptual beauty of how they might be otherwise similar is meaningless to new users and maintenance programmers. - C

Hey, Gary Poster wrote:

On Nov 27, 2009, at 6:32 AM, Martijn Faassen wrote: [snip]

...

So now that we've had some discussion and to exit the "bikeshed" phase,

Wow. That's abrupt, for something at the root of the entire stack.

I realize now that exiting the bikeshed phase was premature. Then again, we don't want to go into circles about APIs forever. Recent discussions were focused on backwards compatibility, so that's progress. [snip]

I am in favor of the above, given a backwards compatibility story that makes existing packages work.

Yay!

...

Utility lookup:

IFoo()

Named utility lookup:

IFoo(name="foo")

Utility lookup with a default:

IFoo(default=bar)

I disagree with this. More below.

[snip]

As a matter of mechanics, when you register something we call an adapter, it is a callable that takes one or more arguments. If we were going to follow the pattern that Marius laid out to establish what happens when, then we have this, roughly:

register callable that takes two arguments: IFoo(bar, baz)

register callable that takes one argument: IFoo(bar)

register callable that takes no arguments: IFoo()

If instead we have the last step as what is proposed here

register non-callable IFoo()

then I think that breaks an important pattern for usage understandability.

I still don't see why that isn't an implementation detail. How we get an IFoo doesn't concern us when we're calling it, as long as we get an IFoo? Even with adaptation a singleton could be returned; it's just the implementation of such would be different. If we take Marius' pattern, registring a singleton directly would simply be a shortcut API for registring a factory for utilities. (Utility factories would make it easier to implement local utilities that aren't ZODB-backed...)

That is, IFoo() can have a semantic if that is valuable, but it is not the same as registering and getting non-called singletons.

What is this valuable semantic? [snip]

(To be complete, I am in favor of ditching "subscription adapters" in favor of other mechanisms related to named singleton lookups.)

I really need to think through subscription adapters; I haven't done any analysis about those.

2) The term "utility" is another barrier to understandability. They are singletons. Explaining them as such is a "well, why didn't you say so" experience.

Another way to explain utilities is that getting a utility is a lot like importing something in Python, except that what is imported is pluggable and the required interface is specified explicitly.

Therefore, I am in favor of removing the necessity to use the word utility. That said, they are not factories. They should not be mixed with the two. My preference for future changes is to have an API using the ``singleton`` name.

"import by interface" to me sounds like it'd clarify matters for more Python programmers. Singleton has all kinds of design pattern connotations that don't really apply here.

Moreover, I think that some of the use cases that Marius referred to for underpowered "utilities" coud be remedied by having a utility/singleton lookup that allowed looking up by required values like the adapter/factory lookup.

I don't understand. Could you rephrase?

...

Features off the table for now -------------------------------

Saying an interface is implemented by a class (Python 2.6 and up) with a decorator we'll leave out of the discussion for now.

It would also be come up with an improved API to look up the adapter *before* it is called, but I'd also like to take this off the table for this discussion.

It seems to me that this, along with the documentation call that Chris gave, is a much more valuable immediate effort. One of the biggest complaints I heard was with debugging. I've spent some thought on the debugging story, and have some APIs sketched out in my experiments--it was one of the first things I worked on. To do it cleanly (the way I envision) would require some work, but a first cut wouldn't be too bad.

Hm, I disagree about what's more valuable. I'd be quite happy when I can grab utilities and multi adapters without having to refer to zope.component all the time. Being able to look up an adapter without calling it in a convenient manner, not so. But multiple efforts can certainly take place in parallel, if we have to volunteers. [snip]

I share Baiju's dislike of inventing __*__ names. What is the necessity? At least __future__ has precedence, I suppose, but Python devs have expressed their opinion clearly now that __*__ is theirs, and I think we should respect it in upcoming decisions.

Ah, I vaguely recalled something about __*__ being theirs now, but wasn't sure.

Finally, per Martin's points, I'm not sure zope.component can actually ever deprecate the old spelling, so I'm not sure __future__ has the right semantic. This is really __alt__ or something, IMO.

I think it can deprecate the old spelling and eventually eliminate it, but it'll take quite a while. It's the deprecation pattern we've used before: * give warnings when someone use it in the deprecated way. * eventually don't allow the deprecated way anymore.

Alternatively, there's Shane's suggestion of a convenience wrapper. I was thinking about that approach too. It has obvious usability disadvantages over having the preferred API immediately available.

Yeah, having to invoke the wrapper would be quite a drawback. [snip]

*Any* kind of backwards incompatibility is a huge deal. I don't find that the simple improvements described here merit any kind of backwards incompatibility. IMO, there needs to be a more compelling argument for that.

Yes, backwards compatibility as I said above is my favorite discussion. That's why I keep bringing it up. I just wanted to frame the debate in such a way so as to consider APIs separately from backwards compatibility and then see how we can get from A to B.

(To be clear, I'm not sure my experiments will turn up changes that merit backwards incompatibility!)

The zope.component API is done, I think. We can experiment with other tools that provide a support shim for it, but breaking it is pretty questionable.

I think having a multi-year deprecation path to do a change is at least a possibility. But it shouldn't be undertaken lightly. [snip]

...

Most importantly, any volunteers?

While I like your desire to move forward, in this particular case I think this speed is rash. I'd also prefer work on debugging tools before these syntax changes, but that's the kind of thing driven by who wants to do the work.

I think there are multiple kinds of speed: a) the speed by which discussions lead to conclusions and a plan of action. b) the speed of implementing the result of the discussion. c) the speed by which any changes will be introduced to users. With step c we should be very careful. With step a, if you don't push towards conclusions and a plan of action, this step is never completed. The debate just fizzles out. So that's why I propose conclusions to see whether we can continue, and to limit the scope of discussions. If people don't like the conclusions they make it loud and clear. :) With step b, if you slow that down too much (by never concluding step a, for instance), you lose all energy and things never happen at all. Regards, Martijn

Am Montag 30 November 2009 16:57:11 schrieb Gary Poster:

As above, I disagree.

As a matter of mechanics, when you register something we call an adapter, it is a callable that takes one or more arguments. If we were going to follow the pattern that Marius laid out to establish what happens when, then we have this, roughly:

register callable that takes two arguments: IFoo(bar, baz)

register callable that takes one argument: IFoo(bar)

register callable that takes no arguments: IFoo()

If instead we have the last step as what is proposed here

register non-callable IFoo()

then I think that breaks an important pattern for usage understandability.

That is, IFoo() can have a semantic if that is valuable, but it is not the same as registering and getting non-called singletons.

Two by-the-ways:

1) The term "adapter" is a barrier to understandability, in my interviews. This is particularly the case when people are introduced to the idea of "multiadapter" and "supscription adapter". In what ways are these anything like a type cast? IMO, they are not. Our usage of adapter is as a factory. Yes, it can be used in other ways--so can a Python class--but that is the essence of how our community uses this technology. Calling all these ideas "adapters" accomplishes nothing. Explaining all of the ideas as "a factory to produce an object that provides the interface" cleanly describes our usage, and both "adapters" and "multiadapters".

(To be complete, I am in favor of ditching "subscription adapters" in favor of other mechanisms related to named singleton lookups.)

One reason I like the syntax proposals for the adapter change is that they treat the interfaces as pluggable factories. This is apt.

2) The term "utility" is another barrier to understandability. They are singletons. Explaining them as such is a "well, why didn't you say so" experience.

Therefore, I am in favor of removing the necessity to use the word utility. That said, they are not factories. They should not be mixed with the two. My preference for future changes is to have an API using the ``singleton`` name. Moreover, I think that some of the use cases that Marius referred to for underpowered "utilities" coud be remedied by having a utility/singleton lookup that allowed looking up by required values like the adapter/factory lookup.

I understand that most of us find IFoo(x, y) looks just beautiful ... and I agree. But the question is, whether that beauty is worth the hassle of the backwards incompatibility and the proposed transition-strategies over the course of *many* years. That's a lot of complication, just to buy some beauty. IFoo is an interface and an interface is at it's core a specification. Lot's of things can be done with this specification: validation, documentation, inspection ... and also lookup and adaptation. For us, adaptation and lookup are the most important uses, but it's not in the very nature of an interface and somebody without zope-knowledge does not neccessarly have that same world view. So it may be convenient to make interfaces callable and return adapters/utilitys and it sure looks nice and requires little typing and all that - but in fact it's a quite zope-ish world view. Wasn't it the main motivation to get rid of the need to having to import and use zope.component whenever we use multi-adaptation or utilitys? So what's so bad about adding methods to interfaces? That meets the original motivation. And it leaves the interface as it's core as a specification and it makes it more clear, what the code does with the interface, instead of imposing our "adapters and utilities are the most important thing about interfaces" attitude onto it. Also, it doesn't mix adapters and utilities conceptually. There is one method to get me a new instance for the interface and the given parameters and another method to get me some singleton/utility. IFoo.instance(x, y) IFoo.instance(x) IFoo.instance() or with less typing IFoo.new(x, y) ... and IFoo.utility() or IFoo.get() or IFoo.single() ... or some other color ... Maybe we want another method, to return the factory without automatically calling it: IFoo.factory(x, y) ... So we could deprecate the interface-calling functionality and just leave it as it is - this way we do not have to worry about year long transitions and confusion everywhere. This is also in line with IFoo.isProvidedBy(x) and the like. OK - just a few thougths from an observer and zope-user. Regards, Matthias