[Grok-dev] solving the grok.template() bug - part A of the solution

[Martijn Faassen]

Hey,

Brandon Craig Rhodes wrote:
> My worry is that we are creating something a bit more complicated than
> it really has to be.  In particular, I note that we are creating a kind
> of scoping that, unless several minutes of thinking have failed to
> reveal it to me, has *no* parallel anywhere in Python itself.  

Thanks for bringing this up. Let me try to give some examples of 
more-or-less parallels in Python.

Class directive inheritance is simple to do in straight Python. First 
the Grok example (look at grok.templatedir for now, not grok.context):

module x:
   import grok

   class Index(grok.View):
      grok.templatedir('foo')
      grok.context(object)

module y:
   import x

   class Something(x.Index):
       pass

In straight Python:

module x:
    class Index(object):
       templatedir = 'foo'

module y:
    import x

    class Something(x.Index)
       pass

We'd expect inheritance to work.

Now let's consider this example of using templatedir as a module-level 
directive:

module x:
   import grok

   grok.templatedir('foo')

   class Index(grok.View):
       grok.context(object)

module y:
   import x

   class Something(x.Index):
       pass

Here we'd like 'Something' to have the templatedir of Index. This is 
what currently isn't working right with inheritance.

In straight Python, you could write something like this:

module x:
    templatedir = 'foo'

    class Index(object):
        def render(self):
            return os.join(templatedir, self.__class__.__name__.lower())

module y:
    import x

    class Something(x.Index):
        pass

You'd expect that to work when you call 'render()' on an instance of 
Something (if I didn't write any bugs in the above :). In Grok it 
currently doesn't.

Of course the parallels with Python aren't complete. This for instance 
should work with Grok:

module x:
   import grok

   grok.templatedir('foo')

   class Index(grok.View):
       grok.context(object)

module y:
   import x
   grok.templatedir('bar')

   class Something(x.Index):
       pass

Now 'Something' would look at 'bar' for its template, not 'foo'. This is 
because module-level directives are equivalent to specifying these 
directives explicitly in all components in the module that don't have 
those directives at class-level.

In order for that to be done with Python constructs, you'd need to 
override the subclass with a custom method.

I happen to like that Martian rule though. And after all, if we wanted 
to do exactly what's available with Python out-of-the-box we'd not have 
no need for directives or the Martian library in the first place.

> That is, if someone wants to understand how directives work - so that they can
> look at code and predict which directive will get used with a class that
> doesn't define the directive explicitly - then we can't say "Oh, it
> works like <such-and-such a normal Python concept>..."; we're creating
> something new with its own rules.

I think the one extra rule people would need to remember is the rule 
above: when a component doesn't have an explicit directive in the class 
scope, look at the module scope to see whether the directive is there. 
The module-level directive is interpreted as being exactly equivalent as 
a class-level directive.

> * Grok first looks for a ``grok.context()`` right on the class.
> 
> * Failing that, Grok does an *inheritance* search, looking through the
>   MRO of the class to see whether one of its parent classes declared a
>   ``grok.context()`` anywhere.
> 
> * Failing that, Grok then tries to do a *lexically scoped* search,
>   looking at enclosing scopes - well, okay, it actually just looks at
>   the module, like old-fashioned Python without true lexical scoping,
>   skipping the levels in between - and seeing if a ``grok.context()``
>   directive exists up there.
> 
> * Failing that, Grok looks in the enclosing module for one, and exactly
>   one, class that implements ``IContext``, and uses that if one is
>   found.

JW already answered this one talking from the perspective of 
implementation. Thinking about it more I get confused again and I'm not 
sure that the implementation JW proposed is enough to cover the rule 
I'll write next. So, I'll rewrite what you have above to what I think 
the conceptual rule should be:

1) Grok first looks for a ``grok.context()`` right on the component,
    explicitly defined there.

2) Failing that, Grok then looks for a ``grok.context()`` in the module
    that the component is defined in.

3) Failing that, Grok looks in the enclosing module for one, and
    exactly one class that implements IContext. Note that this rule
    makes ``grok.context`` special from other directives that
    have class or module scope. The rest of the rule is the same.

4) Failing that, Grok does an *inheritance* search, looking through the
    MRO of the class to see whether one of its parent classes has a
    context. That context can be gained automatically through rule
    1), 2) and 3).

5) Failing that, Grok reports an error during grokking.

You can consider 2) and definitely 3) weird things that Grok does. The 
others follow normal python inheritance and override rules.

Our primary goal in this exercise is to make 4) work correctly. 4) is 
not working correctly primarily because of rule 2), and in case of 
grok.context in particular, because of rule 3). If we dropped both those 
rules 4) would work correctly thanks to Python inheritance (as we stuff 
the explicit directive value into the class).

A good question to ask is why is rule 3) not below rule 4), instead of 
after 2)? I think conceptually the automatic association to a IContext 
should be read as implicitly defining a module-level grok.context. The 
result of this is that a subclass of a view in a module with just one 
model will automatically associate with *that* model. This is preferred 
to associating with the model that the base-view was associated with.

Note that quite frequently grok.baseclass() will be used to mark such 
base views. In that case, grok directives should still be interpreted 
but the view should not actually be grokked and registered and rule 5) 
therefore doesn't apply. This allows someone to write a context-less 
view. If they do put in a model in the same module as the baseclass 
view, the baseclass will however associate with that model still.

> Therefore I advocate that we either drop the inheritance-search, and
> have ``grok.context()`` searched for like the ``x`` in ``x.y``, or drop
> the module-level default, and have it searched for like the ``y``.  Our
> rules are already going to be more complicated because we have the
> fallback to looking for ``IContext``; I don't think we need to be adding
> a third layer.

Here you propose either dropping rule 4) or rule 2).

Dropping 4) means no more inheritance of directives. I think this breaks 
fundamental expectations a Python programmer has.  It's also a rule 
zope.interface.implements() (aka grok.implements()) follows. Think about 
grok.implements and tell me what you expect in case of *that* directive. 
I think dropping rule 4) is therefore out of the question.

Dropping 2) means no more module-level directives. This would work in 
any case besides that of ``grok.context()``, as we'd also need to drop 
3). (which you don't propose).

If we want to make all rules work, we'd need the directive value lookup 
procedure to do the following:

* check whether the value is defined on this class (and not on any base
   classes, check only dict of this class). If so, we're done.

* Check whether the value is defined in the module of this class. If so,
   we're done. In case of grok.context we extend this rule and also look
   for a non-ambiguous IContext in the module of the class.

* Check whether the value is defined on the class, including any base
   classes (normal getattr on the class for the directive value). If
   so, we're done.

* Check whether the value is defined on the module of the module of
   the baseclass of this class (for each baseclass, following MRO). If
   so, we're done. In case of grok.context we also look for IContexts
   again.

I hope I got the rule right this time, whenever JW and I talk about it 
we both get confused. :) Note that the rule above suggests some way to 
hook in the grok.context behavior into the process, which is nice.

Now if we dropped the grok.context auto-association rule and we dropped 
module-level directives, it'd become much simpler to implement everything:

* just get the value from the class.

I think that's too big a change to drop on people though. Besides, I do 
use auto-association and module-level directives quite frequently in my 
code and they're quite nice to have and I don't think usually make 
things much harder to read either.

Regards,

Martijn