[Zope-dev] Lazy expressions appear to cause memory leaks

[Geoff Davis]

In Zope 2.8+ there is a little known but very useful TALES feature: you
can have expressions be lazily evaluated.  For example,

<span tal:define="foo lazy:python:someExpensiveMethod()" />

The "lazy:" prefix causes the python expression to not be evaluated until
foo is used somewhere.

There appears to be a fairly big problem with this setup.  The lazy:
prefix wraps the expression in a LazyExpr which stores the expression and
its context.  The LazyExpr in turn generates a LazyWrapper (that
holds similar information) which ends up getting held in the
TALInterpreter's global/local variable list.

The expression context for TAL expressions in a page template includes
things like the template itself, the context object (a fully wrapped
object), the request (chock full of complicated stuff), and so on.  It
appears that storing the expression context in the lazy wrapper creates
some kind of circular reference or something similar that is preventing
garbage collection of these lazy wrappers.  The result is a nasty memory
leak.

The problem appears to be fixable via some cleaning up in
PageTemplate.pt_render after the interpreter does its thing.  The code
snippet below is probably overkill, but something like this appears to be
what is needed:

    context = getEngine().getContext(c)
    TALInterpreter(self._v_program, self._v_macros,
                   context,
                   output,
                   tal=not source, strictinsert=0)()

    # clean up - try to eliminate circular references - this may be overkill
    context._compiler = None
    context.contexts = None
    context.repeat_vars = None
    from Products.PageTemplates.DeferExpr import LazyWrapper
    for k,v in context.global_vars.items():
        if isinstance(v, LazyWrapper):
            v._expr = None
            v._econtext = None
            v._result = None
    if context.vars:
        while len(context.vars):
            context.vars._pop()
    context.global_vars.clear()
    context.global_vars = None
    context.local_vars.clear()
    context.local_vars = None
    context.vars = None
    context._scope_stack = None