[Grok-dev] solving the grok.template() bug - part A of the solution

[Brandon Craig Rhodes]

Jan-Wijbrand Kolman <janwijbrand at gmail.com> writes:

> With only slow progress I'm still working on the grok.template()
> issue.  Recently Martijn and I had a couple of hours to spend on this
> problem.  We found there to be several parts to a solution.

If gainful employment does not intervene, I will try to actually look at
the code surrounding this issue tonight.  But for the moment, I wanted
to think about the concepts a little.

My worry is that we are creating something a bit more complicated than
it really has to be.  In particular, I note that we are creating a kind
of scoping that, unless several minutes of thinking have failed to
reveal it to me, has *no* parallel anywhere in Python itself.  That is,
if someone wants to understand how directives work - so that they can
look at code and predict which directive will get used with a class that
doesn't define the directive explicitly - then we can't say "Oh, it
works like <such-and-such a normal Python concept>..."; we're creating
something new with its own rules.

In Python, scoping and inheritance seem to be kept completely
orthogonal.  When the expression ``x.y`` is evaluated, the rules by
which ``x`` is discovered are simply (and completely!) different from
the rules by which ``y`` is discovered.  The ``x`` will be searched for
using scoping, and scoping *only*: first in the current scope, then the
next outer scope, then up at the module level if it's not found there.

    SIDE NOTE: the ``x`` will actually *not* be searched for if the
    enclosing scope is a class, oddly enough, so that the value printed
    in each of the following two cases is different, which used to
    confuse me until I learned the rule well:

        x = 3
        class C(object):
            x = 4
            def printx(self):
                print x

        >>> c = C()
        >>> print c.printx()
        3

        x = 3
        def c():
            x = 4
            def printx():
                print x
            return printx

        >>> printx = c()
        >>> printx()
        4

    END OF SIDE NOTE.

Okay, back to ``x.y``: while ``x`` is searched for using scoping, ``y``
is searched for using only inheritance.  It could be an attribute on the
``x`` object itself; or on the class of ``x``; or on one of superclasses
of that class, which are faithfully searched in MRO order until the base
class is finally reached.  If the search fails (and no ``__getattr__``
mechanism kicked in to handle the lookup in some different way), then an
error is simply returned; Python does *not* then try looking in any
enclosing scopes for the value, because that would be swapping to a
completely different kind of search.

Now it seems to me like our CLASS_OR_MODULE directives get these two
clean, distinct Python lookup rules, and mush them together: unlike
Python, we *do* try both, which means that a programmer reading a class
without, say, a ``grok.context()``, has *three* different places they
need to look for the context!  When I was a young Grok programmer, I was
taught that "If ``grok.context()`` is missing then Grok, which favors
convention over configuration, looks through the current module for an
appropriate context"; but this was far short of the truth!  It looks to
me like the rule is (or is supposed to be):

* Grok first looks for a ``grok.context()`` right on the class.

* Failing that, Grok does an *inheritance* search, looking through the
  MRO of the class to see whether one of its parent classes declared a
  ``grok.context()`` anywhere.

* Failing that, Grok then tries to do a *lexically scoped* search,
  looking at enclosing scopes - well, okay, it actually just looks at
  the module, like old-fashioned Python without true lexical scoping,
  skipping the levels in between - and seeing if a ``grok.context()``
  directive exists up there.

* Failing that, Grok looks in the enclosing module for one, and exactly
  one, class that implements ``IContext``, and uses that if one is
  found.

I don't want to trod on any toes, but it really looks to me like we are
grasping for too much writer-convenience here, and not nearly enough
reader-convenience.  We are trying, through all of these rules, to
eliminate a few ``grok.context()`` calls that developers might have to
make when first writing their app; but in return, readers of the code
encountering a class without an explicit ``grok.context()`` have *three*
places they need to look - and they have to remember the order! - in
order to find what the context really is.  I think that this is an undue
burden, and that we should have simpler rules that sometimes make people
be more explicit, rather than this  complicated  stack of rules that
sends readers hunting all over the place.

Therefore I advocate that we either drop the inheritance-search, and
have ``grok.context()`` searched for like the ``x`` in ``x.y``, or drop
the module-level default, and have it searched for like the ``y``.  Our
rules are already going to be more complicated because we have the
fallback to looking for ``IContext``; I don't think we need to be adding
a third layer.

My vote would be removing the MRO search because (a) it sends the poor
reader searching through module after module to read all the parent
classes, which it's always hard to read in the right order if multiple
inheritance is used; and (b) a module-level ``grok.context()`` kind of
"looks like", and has a pleasant symmetry with, the search at the module
level for an ``IContext`` class if one is present, so that if a class
does not define ``grok.context()`` then the answer is somewhere out in
the module one way or the other.

Of course, this would leave in place the problem that Martijn and
Jan-Wijbrand are working on; but since their question seemed to raise
the larger issue, I wanted to go ahead and ask about it.  Remember... :-)

    Explicit is better than implicit.
    Simple is better than complex.
    Complex is better than complicated.
    Flat is better than nested.
    Sparse is better than dense.
    Readability counts.

-- 
Brandon Craig Rhodes   brandon at rhodesmill.org   http://rhodesmill.org/brandon