[Grok-dev] towards a new publisher

[Martijn Faassen]

Hi there,

Here are some of my thoughts about working towards a new publisher for 
Grok (or the Zope Toolkit as a whole). They're not completed but I 
thought by sharing some I might get some people who are interested in 
contributing.

What is the publisher? I'm using it in its fairly broad sense:

* it's the thing that handles the request

* it then looks at the path and resolves it somehow to a view

* the view result is returned as the response

This includes things like request/response objects, URL traversal (or 
routing), and view lookup.

Why should we replace the publisher?

The old publisher is ravioli code: it's scattered over a rather large 
range of code and is hard to follow. It's very flexible in some ways, 
but perhaps not in the right ways. It's hard to keep in ones mind. We 
want to do stuff we do now in a nicer way, such as REST or traject, 
which uses some hacks inside. We want whatever happens now to be faster, 
ideally (though I don't think it's a major drag on performance). We want 
the code footprint to be smaller. We want better documentation for what 
it does. We want it be less intimidating to hack on. We want it to be 
more comprehensible. We want to do new cool stuff like having view 
conditions or predicates ("this view should only show up if it's looked 
up on an object like this, or when you're in this layer, or when the 
user has this permission, or when the request is a GET request, etc").

Okay, cool, what needs to be done?

Three things:

* create a new publishing environment

* have an upgrade path from the old environment to the new one

* lose the old publisher codebase.

I'll handle these in reverse order, and I'll start with losing the old 
publisher codebase. While it's not *necessary* to lose the old publisher 
codebase, if we don't, we'll have a lot of distracting duplicate code 
around. That's a job for dependency refactoring. Now that we've got the 
main dependency refactoring for the ZTK out of the way, it becomes time 
to look at what depends on zope.traversing and zope.publisher and such 
and see whether we can loosen the dependency sensibly. It'll be tricky, 
but we can do this step by step on a case by case basis ("okay, let's 
look at zope.container and how can we detach it from zope.publisher? 
putting the traversal adapters in another package would probably do it.. 
but what package?"), until we can cut the publisher out.

That's all eventually; we need a way to get to a new publishing 
environment without breaking the world. So we likely end up with hybrid 
Grok environments which still have the old publisher in it. That's the 
upgrade path; allow people a way to switch over a working codebase to a 
new publisher, with as small an impact on their code as possible.

Now as to the new publishing environment. I'd like to do what other 
environments already do, be WSGI friendly and use WebOb for the request 
and response object.

What would the new publishing environment look like?

It probably looks a lot like BFG. Can it be fully BFG? I don't know. If 
not, we'd definitely need to steal a lot of ideas from BFG. We'd need to 
do some work to support various patterns Grok supports now. Perhaps we 
can get far by plugging in a custom traversal mechanism. I'd also like 
the system to be location-aware and use view lookup on models, even when 
using routing, and traject can do that (and is reusable). I think that 
model and location orientation is something that makes Zope environments 
special and we need to build on that.

This is a big job, and it's going to take a long time to accomplish. I'm 
going to work on it, but I don't know when or even if I'll be done. I 
think it's important for the future of Grok to do it, however.

Thoughts?

Regards,

Martijn