[Grok-dev] Re: Grok component reuse in the Zope 3 world (was Re: Re: ANN: megrok.quarry)

[Martijn Faassen]

Hey Gary,

[meta discussion about the debate]

Thanks for replying back. I think this discussion has helped both you 
and myself to articulate some points better, and hopefully has now 
communicate some of these points to any readers we still have left on 
this thread. :)

Gary Poster wrote:
[snip]
> So, the rest of my reply, and debate, is out of politeness, really, 
> since you have taken the time to think and write back. :-)  My goal has 
> been met as best as I had hoped it to be met, realistically.  You know I 
> don't stick my head up in the community too much, and a big reason is 
> that I don't have much time at home or work for it.  I respect your time 
> too, so I want to "talk back".  But, heh, if you don't give another 
> thought-provoking reply in return I won't mind. ;-)

Thanks for the appreciation! I think we understand each other better 
now. I think so because my reply is more about details and explanations 
than about debating anything. :)

[snip]
>> * I think the problem might in fact be smaller than you suspect. ZCML 
>> statements produce configuration state, just like grokking does. You 
>> should be able to produce with ZCML statements whatever registrations 
>> that Grok can produce through grokking. You would still need to depend 
>> on base classes provided by Grok, though.
> 
> So, Grok uses the underlying configuration machinery that ZCML uses, 
> with two steps so you can override registrations and detect conflicts 
> and so on, and Grok and ZCML manipulate the same data structure in the 
> first stage, and therefore ZCML can override Grok registrations and vice 
> versa?  If so, hooray.  That would allay many fears.

Unfortunately, not yet. I can however truthfully say this has been our 
plan since the very beginning. :)

We haven't done this yet out of expedience - we didn't want to spend a 
lot of time up front to make this happen while we didn't really know yet 
what Grok would look liked from a higher level perspective. So, right 
now we directly use the component architecture's APIs. I think that this 
  was the right decision at the time.

I also think we know enough now to start the process of getting this 
done now, though. This means we may need to do some work on the Zope 3 
end, to produce a python API for issuing configuration actions, and 
possibly get ZCML to reuse that same API. Possibly we can get away with 
doing less. We haven't explored the problem enough yet to know, but we 
are bound to find out when we try.

> One that would not be allayed is that, where ZCML allows separate files 
> so you can mix and match what you want to import and use, AFAIK you can 
> either grok a package or not.  But that's a pretty trivial complaint if 
> you are using the underlying ZCML configuration machinery.

Right, I hope we can take care of these concerns by other mechanisms 
(such as tagging).

> I snipped out a lot above and below because the above is basically my 
> response to big chunks of it, and I was repeating myself.
> 
>>>> Is it because a grok-based component wouldn't include enough 
>>>> security declarations? (because in grok you don't need to). That's 
>>>> the only thing I can think of. It's a hard one to solve, as writing 
>>>> code with a lot of permissions sprinkled around is a major pain Grok 
>>>> tries to get rid of.
>>> Certainly that's an issue.  Especially given that security changes 
>>> can be very important to us, as I alluded above.  If grok security 
>>> assumptions pervade a bit of code, it's almost certain that I can't 
>>> use it.
>>
>> I'd argue it's the other way around - normal Zope 3 usage has security 
>> assumptions spread throughout the code.
> 
> Certainly there are security assumptions spread throughout the code that 
> are different than the Grok approach: Grok wants to do security on 
> view-level, and everything else in Zope 3 doesn't. 

And quite a bit of Zope 3 code is security-neutral - whenever you use a 
utility for instance.

> The Grok approach is 
> appealing for apps that don't have TTW code, and in retrospect might 
> have been a reasonable way to structure the default Zope 3 
> configuration.  Amusingly (to me anyway), though, I actually want the 
> current Zope 3 approach, because I actually have TTW code in some of my 
> apps.

Cool, TTW Zope 3 code, that's the first time I've heard of it being in 
practical use! :) I'm sure Zope 3 developers would be curious to find 
more about this!

> Doesn't effectively Grok make the same kind of security assumptions, but 
> in reverse?  If not, interesting and cool. 

I'm not sure what you mean by this question. We basically only do a 
security check when a view is requested. That is of course a security 
assumption, but not one that tends to be very infectious in code.

> Maybe we ought to make that 
> switch easier to make in Zope core, then.

That would be nice to have. At present we use a hack that keeps turning 
it off during traversal, though Philipp and Christian can tell you more 
about this than I can.

> As you might expect, though, 
> practically, I need it to be easy to keep on using the current Zope 3 
> approach.

Understood.

> Of course, if the Grok-like changes to Zope 3 core would mean, for 
> instance, that objects without security assertions would be assumed to 
> be playing the Grok way, and so implicitly/automatically allowed, that 
> would not be so good for me.

 > The kind of security assumptions I was thinking of in my original
 > comment was assuming a security policy.  Maybe Grok's stuff wouldn't
 > make that a problem, but because AIUI you have changed the security
 > story it would definitely be a task to confirm that.

Grok just rips off security proxies so we don't have to deal with any 
Python-level security policy. We want something similar to Zope 2, where 
filesystem code is trusted.

The security policy with default Grok hasn't changed, just the policy of
introducing security proxies.

We only want the 'grok' package take care of getting rid of security 
proxies. When you develop a grok-based component and want to play nicely 
with non-Grok code (which you should) you'd not depend on the 'grok' 
package directly, so your package would be neutral to security policy.

Two possible problems with this:

* you are not going to make permission declarations in most places. This 
would be a problem for zope 3 code which needs those permissions. 
Hopefully ZCML can be used to introduce them externally where needed.

* Code will make the assumption that you can call anything without 
security getting in the way. This might mean that code is structured in 
a way hard to provide the correct permissions for. Also, obviously no 
removeSecurityProxy anywhere. We can't really require of developers to 
worry about python-level security just for compatibility's sake, as that 
would defeat the point of trying to take away some of those worries from 
them, so this one is hard to solve. Hopefully it doesn't occur too often.

[snip]

>> The problem is that what you're asking is not really doable and have 
>> Grok still be Grok, as I described above in the case of adapters. I 
>> can't separate the adapter registration from the adapter 
>> implementation with Grok:
>>
>> class MyAdapter(grok.Adapter):
>>    grok.context(ITheAdapted)
>>    grok.provides(ISomeInterface)
>>
>> But if you wanted to use ZCML to register this adapter, you'd write 
>> something like:
>>
>> <adapter for="ITheAdapted" factory="module.MyAdapter" 
>> provides="ISomeInterface" />
>>
>> and that should still work.
> 
> [aside]
> 
> In that particular example, isn't that a bit of wheel reinvention?  
> These days, I can do the following:
> 
> class Foo(object):
>     zope.interface.implements(IFoo)
>     zope.component.adapts(IBar)
> 
> and then
> 
> <adapter factory="Foo" />

zope.component.implements will also work, if Foo implements only a 
single interface.

The other wheel reinvention (grok.context instead of component.adapts) 
happens because it's part of the heart of Grok. You can leave out 
grok.context and have it work too, if what you're adapting is a content 
object defined in the same module, or if grok.context() is used on 
module level. Part of the 'convention over configuration' story.

Note that grok.context is not exactly the same as component.adapts, as 
grok.context in the case of views just points to the content object and 
hides the multi adapter details.

That said, we should look into making zope.component.adapts work too.

> or similarly
> 
> @zope.interface.implementer(IFoo)
> @zope.component.adapter(IBar)
> def foo(bar):
>     # ...
> 
> and then
> 
> <adapter factory="foo" />
> 
> Couldn't grok reuse the standard zope.interface and zope.component 
> spellings?

It should at least allow those spellings. I'm a bit worried we'd end up 
having multiple ways to do it, but I think it's worth it in this case, 
if we don't give these spellings too much prominence in the tutorial.

> [...again, snipping bits to which I think I effectively replied...]
> 
>> You're clearly not forcing us to use ZCML instead of other approaches 
>> for our own packages, otherwise we couldn't have written Grok. Why do 
>> you think we would be forcing you if you turn this around and you use 
>> a Grok-based package?
> 
> Well, because, (as I think you acknowledge several times) "you aren't 
> there yet".  As I said at the start, I know you guys are smart, and 
> since you are working towards this, then yay.  As I said, if you are 
> using the underlying Zope configuration stuff that ZCML uses, then many 
> of my most significant worries go away.

Yes, good point of course; we're not there yet. The eggification of Zope 
3 made us realize we need to get there relatively soon. Some of the 
spiritedness of my response can be explained because I was already 
hacking away on martian, which is an important step on the way. It's a 
concern I've been thinking about a lot recently. :)

> Thanks again for your reply.  I'll try to make a quick reply to your 
> next one too, from my response to Lennart. ;-)

And thank you for keeping up the discussion. It's good to have this 
exchange of views and concerns.

Regards,

Martijn