[Grok-dev] Re: Grok component reuse in the Zope 3 world (was Re: Re: ANN: megrok.quarry)

[Martijn Faassen]

Hey Gary,

Even though I argue, I must thank you for bringing this up. One the one 
hand your insistence that you don't want to use alternative 
configuration mechanisms makes me less motivated to do the work I had 
planned, as apparently it will have no effect. On the other hand, it's 
good to know this in advance so I won't put my hopes up too much. I will 
still do it as this was about scratching my own itch more than anyone 
else's anyway.

Gary Poster wrote:
[argues ZCML is an essential ingredient towards creation of reusable 
components and that argument is useless]

I think we should agree to disagree on this one. :)

[Grok "combatting" reuse outside the community]

I think the word 'combat' is rather too heavy a term. There's a concern 
and I hope we can fix the concern, though I don't think we can fix it in 
the way you want us to.

[snip]
>> Why do you consider it insufficient? In a package that uses grok, I 
>> can declare an adapter. Same with ZCML. What's the difference?
> 
> If you write good Python code I want to reuse, and I don't want to use 
> grok (or have my package "grokked"), I'd like to be able to use it 
> without depending on grok.  That's the heart of it. 

Do I understand you correctly and do you think *your* package will be 
grokked if you use a package that is grokked? That's certainly not true. 
Grok is not a virus that infects other packages.

If that's not what you meant, you have to have a reason for not wanting 
to depend on Grok, or to be more correctly, depend on a smaller 
configuration library that Grok will be using.

> "Martian" appears 
> to be about being able to disseminate the grok approach along with your 
> grok packages.

Yes, the idea is that if you want to use a package that uses Grok-based 
registrations, you just include it in your dependencies. That package 
will use martian and things like grokcore.component instead of 
zope.configuration and zope.app.component.

> The issue I'm trying to raise is the opposite: what if I 
> don't want approach but I do want to share your code.

I consider the following to be Grok's problem:

To make it as easy and non-intrusive as possible for you to reuse a 
package that uses Grok-based registration instead of ZCML, within a 
larger application that doesn't use Grok in any way.

Now as to what I consider to be your problem:

If I understand you correctly, you say you still don't want to use 
packages that do registrations in any other way than through traditional 
ZCML (instead of the ZCML 'grok' directive). You want to use ZCML with a 
grok-based package.

Let's distribute responsibilities for accomplishing this:

I am willing to act on specific suggestions as to how make it easier for 
you to use ZCML registrations on Grok-based code. If Grok is somehow in 
the way while it'd be fairly easy to fix that, I think we should fix that.

I do not consider it to be my responsibility to restructure my 
Grok-based code to make it more easy to use with ZCML. Two reasons:

* Grok actually has particular ways to structure code that this could 
interfere with. It is not feasible to take the adapter registration away 
from the adapter, for instance. The adapter class *is* the registration 
information.

* I think the problem might in fact be smaller than you suspect. ZCML 
statements produce configuration state, just like grokking does. You 
should be able to produce with ZCML statements whatever registrations 
that Grok can produce through grokking. You would still need to depend 
on base classes provided by Grok, though.

> For instance, what if you had written hurry.query for Grok.  It's a nice 
> API.  What if I wanted to use it, and I didn't want to use Grok.  All 
> I'm asking is that grok encourage their community developers who are 
> making packages that are generically useful, like hurry.query, to do it 
> in a way that the larger Zope 3 community can use them without having to 
> use grok.  There are several ways to accomplish this, I'm sure.  I 
> suggested a couple.

I have the impression you consider grok far more "all or nothing" than 
it really is, and that this contributes to your wish not to use packages 
that use grok-based registrations, but perhaps I'm mistaken.

As far as I understand it, you're saying you don't want to use any 
packages that use something else than ZCML for component registration. I 
could say the same, and refuse to use packages that use ZCML for 
component registration. I don't say this, because I think that'd be 
shooting myself in the foot. :)

In by far the majority of cases I don't care *how* component 
registrations happen, I just care *that* they happen. How it's 
accomplished is an implementation detail of the package in question. I 
just care that I can do IFoo(bar) and get an adapter that implements IFoo.

I know there is a small amount of cases where one does care about 
*which* registrations happen, and one wants more control over it. I hope 
we can come up with solutions where we can offer that control for 
Grok-based packages (including ones that are usable from ZCML).

>> Is it because a grok-based component wouldn't include enough security 
>> declarations? (because in grok you don't need to). That's the only 
>> thing I can think of. It's a hard one to solve, as writing code with a 
>> lot of permissions sprinkled around is a major pain Grok tries to get 
>> rid of.
> 
> Certainly that's an issue.  Especially given that security changes can 
> be very important to us, as I alluded above.  If grok security 
> assumptions pervade a bit of code, it's almost certain that I can't use it.

I'd argue it's the other way around - normal Zope 3 usage has security 
assumptions spread throughout the code. Grok doesn't do security except 
on the view-level. This means we don't care about assigning permissions 
in many cases. If you want to use a Grok-based package and care about 
content-level security, you will have to provide external ZCML to state 
your permissions. I think that would probably work out of the box, but I 
haven't tried.

[snip]
>> It's you who says Grok is about applications and not components, not 
>> me. :)
> 
> Hm.  OK.
> 
>> Grok should also be about components, because otherwise there is no 
>> easy way to spin off components from applications. This is a very 
>> important mechanism of component generation. In my opinion it would 
>> defeat the purpose of Grok if people would have to go back to ZCML as 
>> soon as they want to spin off a component. Spinning off a component 
>> should be moving code into another package, and the addition of ZCML 
>> shouldn't be required.
> 
> I don't want grok developers to *have* to make some ZCML; but I don't 
> want grok developers to make nice components that exclude those of us 
> who continue to choose to use zcml.

I think that is possible right now. If you are willing to dive into a 
Grok-based package and want to write a lot of ZCML registrations to 
replace the information in the code then I think you can do that.

It could even be automated - it is possible to write code that sniffs 
Grok's registration information and produces ZCML. The information is, 
of course, all there (excepting information we don't need, such as 
permission information).

I'd argue if you go as far as generating ZCML you can just as well go 
the whole way and simply use the ZCML 'grok' directive on the package.

Full disclosure: some aspects of Grok may be a bit more tricky to 
represent in ZCML as they depend on the event system to install, say, 
the catalog, into an application. I'd say those are usually more 
application-oriented and less component oriented, so typically won't 
matter as much for components. This is an area to watch carefully, however.

>> Grok is about making the component architecture easier to use. I don't 
>> want to write code, applications *or* components, where I need to go 
>> to separate ZCML files for registrations. I considered it a pain after 
>> having tried it with a large application, before working with Grok. 
>> Now that I've used Grok, I *really* don't want to go back anymore. :) 
>> I also want other people to use Grok-based components, which is why we 
>> got a plan. :)
> 
> I know that's how you feel, I know you speak from experience, and I 
> respect your opinion.  All I'm saying is, to the grok community, please 
> don't force other Zope 3 micro-communities to agree with you in order to 
> you use your *potentially sharable* cool code.

I understand and share that goal. I want to be able to create a packages 
that both you and me can use.

I'm willing to go quite far in ensuring that a package that uses 
Grok-based configuration can be used within a Zope 3 application that 
doesn't use Grok anywhere else. I think this is an important goal.

I don't see the fact that a particular package uses ZCML as forcing me 
to agree with ZCML-based registrations in my own code. Similarly I don't 
think you should be seeing a package that uses another registration 
mechanism as forcing you to agree with that way of registering 
components in your own code.

You appear to be arguing that it does, and say you actually won't use 
such packages. If these packages are entirely compatible in a technical 
sense (something I'm willing to work hard at ensuring), I'd argue that 
it's your own opinion that's forcing you there, not mine.

>>> Second, when the first approach is inappropriate for whatever reason, 
>>> consider putting the grok glue code in a separate module.  That way 
>>> someone can import your generally interesting code without depending 
>>> on grok, or the grok approach.  Perhaps someone else will even add a 
>>> zcml file, and folks can choose to go the grok road or the zcml road 
>>> for your package.
>>
>> All grok packages already have a configure.zcml file. It just contains 
>> a single line, telling it to grok the package using the only ZCML 
>> directive defined by grok, namely 'grok' :). The result of including 
>> that configure.zcml is the same - component registrations. This is why 
>> we are already close to reusability of grok-based code in Zope 3. Two 
>> reasons why more needs to be done:
> 
> I was trying to be constructive when I suggested that you divide up your 
> Python grok hints.  That would let you do the kind of division I said I 
> (we) do with multiple zcml files.

I know your suggestions are well intentioned.

The problem is that what you're asking is not really doable and have 
Grok still be Grok, as I described above in the case of adapters. I 
can't separate the adapter registration from the adapter implementation 
with Grok:

class MyAdapter(grok.Adapter):
    grok.context(ITheAdapted)
    grok.provides(ISomeInterface)

But if you wanted to use ZCML to register this adapter, you'd write 
something like:

<adapter for="ITheAdapted" factory="module.MyAdapter" 
provides="ISomeInterface" />

and that should still work.

> Otherwise, all you are telling me I 
> have is a big switch: grok, or not grok.

No, I'm not telling you that. I'm telling you that you should be able to 
take a package that uses Grok to do its configuration registrations and 
use it in an application that does not. We need to do some work on 
Grok's end to accomplish this.

I'm also telling you that if you don't use a configure.zcml of a grok 
package, you'll get quite far in writing the equivalent ZCML. I think 
that's not a very good approach, but you could.

I'm telling you that what we can't do is get rid of grok's base classes 
so you don't have to depend on those anymore. So, for instance, I can't 
put the adapter implementation in one file, and the registration in 
another, without hopelessly overcomplicating the code.

With a little bit of hassle what we *can* do is the following:

class MyAdapter(grokcore.component.Adapter):
    grokcore.component.context(ITheAdapted)
    grokcore.component.provides(ISomeInterface)

that's a pain to write. Hopefully we can get away with putting the 
following on top of a module:

from grokcore import component as grok

That way, the package will hopefully only depend on grokcore.component 
and martian, and not grok itself.

[snip]
> My contention is that the grok approach makes some sense for 
> applications, but is not friendly for sharable packages.

This is where we will have to agree to disagree.

>>> In sum, I laud the Grok community's energy and desire to make cool 
>>> Zope 3 technologies easier to begin with and use.  But I encourage 
>>> you to direct some of your energy to remembering that one of the cool 
>>> Zope 3 ideas is component reuse.  Don't code yourselves away from the 
>>> rest of us: we want you around. :-)
>>
>> I find it somewhat ironic that you say this in response to a message 
>> where I talk about how we are going to tackle exactly this problem. :) 
>> Not in the way you envision, perhaps - ZCML won't come back in play, 
>> but I'm pretty certain the sketched out approach will solve this problem.
> 
> So, I believe you are saying "I'm already solving this problem by 
> forcing everyone who wants to use our code to not use other approaches 
> (like ZCML)".  Perhaps I'm overstating it for drama, but do you see the 
> kernel of truth?

I think we should have an even playing field of the component 
architecture. I think we agree on that principle, though not on how to 
accomplish this.

You want an application to use a single configuration mechanism for all 
the packages it depends on. You're in this situation now, and the 
configuration mechanism is ZCML.

I'm arguing for a situation where you can use a mix of packages using 
different configuration mechanisms. With Grok we have that situation 
now, and the configuration mechanisms are Grok and ZCML.

You're clearly not forcing us to use ZCML instead of other approaches 
for our own packages, otherwise we couldn't have written Grok. Why do 
you think we would be forcing you if you turn this around and you use a 
Grok-based package?

Regards,

Martijn