[Grok-dev] a nice research project

[Martijn Faassen]

Hey guys,

This is going to be long as it contains thoughts I've been having for a 
while that I probably should have been sharing more explicitly.

My first response to any comparison where Grok comes out badly is to 
find excuses for Grok. The main excuse that will always work is that 
this is just *a* benchmark and if you just adjust what you look at then 
Grok will do better. And of course if someone advocating another 
framework does the benchmark, it's likely that they'll pick a benchmark 
which makes their framework comes out best. I'd do the same. :)

(...man had I trouble finding one when cElementTree came out while lxml 
was just newborn!)

It also turned out that when Chris adjusted the benchmark suddenly a lot 
of the weird ZPT and XML-related stuff went away and Grok is doing a bit 
better. That's good - I'm still confused about where that came from 
initially, but at least the page template engine wasn't operating during 
almost all of the test (except possibly the first request which is now, 
I take it, unmeasured).

My second response to any comparison where Grok comes out badly is to 
look at Grok itself and search for ways to make Grok better. We're 
certainly not at the end of improvement here. Let's acknowledge Grok can 
be improved here. Let's learn.

I got a head start in this, as I've been looking at the repoze.bfg 
project for a while (from a distance). So Chris's blog wasn't 
particularly news to me though it does put things starkly (he got to 
pick the benchmark). There are, in my mind, two main things Grok can be 
envious about compared to repoze.bfg:

* much simpler dependency structure (therefore bfg is easier to 
understand and less intimidating to install)

* a much simpler and more comprehensible publication process.

The dependency structure bit has been long known as an issue in the Zope 
framework, since it got split up into little pieces. Seeing repoze.bfg 
only highlighted this. So we had a sprint recently to help clean up 
dependency structure, which I also see as a prerequisite for improving 
the libraries themselves. We've made good progress on it, though tons of 
work remains to be done; feel free to pitch in.

Improving the publication process in the Zope framework and Grok is a 
next step. Grok (and the Zope framework) are indeed doing a lot to get a 
page published. In part this is because the Zope framework tries to be 
flexible and pluggable in the publication process in ways that other 
frameworks care less about. Some of this flexibility is useful, and much 
of it is quite possibly less useful. To make things worse, the Zope 
framework publication procedure *is* scattered around the codebase, 
which makes it rather difficult to understand and modify.

I'm not the only one interested in this stuff:

http://shane.willowrise.com/archives/repozublisher/

Grok is not a new web framework, like repoze.bfg. It builds on the Zope 
framework, and tries to stay compatible with it. We believe the Zope 
framework has a lot to offer. It's mature and the codebase of individual 
components is typically high quality, and there are a lot of extensions.

This year, it's time we (Grok developers) go back to the underlying Zope 
framework to improve its comprehensibility. We're a major consumer of 
the Zope framework after all, so we have that responsibility (and 
*need*). That means a better dependency structure, less code shipped 
that isn't actually used much and that we can do better at anyway (the 
ZMI), and improvements to the publisher system, among other things.

I understand all emotions in this discussion. Our task is to take the 
emotions and channel them to productive activities. We learn and 
improve. Let's go for it.

Regards,

Martijn