Exposing a the simple publisher (Re: [Zope-dev] Re: Reducing dependencies of zope.publisher)

[Jim Fulton]

On Mar 25, 2008, at 5:25 AM, Martijn Faassen wrote:

> Jim Fulton wrote:
> [snip]
>> Thoughts? Objections?
>
> A simpler publisher has been on my wish-list for a long time now.
>
> I'm a bit worried though that a publisher born from the current Zope  
> 3 publisher with the goal to build up enough support for the Zope 3  
> publisher to make use of the code will not in fact be simpler. The  
> current Zope 3 publisher supports quite a few pluggability points  
> (though not necessarily always in the most convenient places), and  
> has a lot of interacting components, which makes it rather hard to  
> comprehend sometimes.

I suspect you are confusing, as the repoze folks seem to have, the  
publisher, with the way that the Zope 3 server set up code wires it  
up.  When we originally set up zope.server at the dawn of Zope 3, we  
inadvertently created a gordian knot of interacting components. When  
we added WSGI and Twisted support, we just made this worse by adding  
additional threads.

The basic publisher framework is pretty straightforward.  There is a  
generic publisher and a publication component that takes care of  
customizing the publisher for a particular application.  The  
publication interface, zope.publisher.interfaces.IPublication, defines  
the available hooks. The recent addition of zope.publisher.paste makes  
selection of a custom publisher much simpler -- as well as supplying  
paste support.  Unfortunately, I don't think anyone has paid attention  
to this. :(

If all you want is more control over how the publisher works, I think  
the current publisher simplified via the new paste support should meet  
your needs.  It's possible some folks need more hooks, but then we  
should focus on evolving the publication API,  
zope.publisher.interfaces.IPublication.

> You seem to be confident you can create this simplicity while  
> retaining enough pluggability to build the Zope 3 publisher back up  
> on it, though, correct?

Yup. In fact, all I'm planning to do is to move most of the existing  
publisher code to a separate package sans interface declarations,  
xmlrpc, and a few other things that many applications don't need. I  
intend to keep the current publisher+publication framework.  I don't  
expect to rewrite any code, other than tests, or change any  
semantics.  In fact, I'd like migration from the leaner pubisher to  
the full publisher to be as easy as in import change.

> I'd be interested in using such a publisher as the foundation for  
> the publisher in Grok.

Why? (But read to the end before answering. :)

> Of course Grok does currently use the current Zope 3 publisher in  
> Zope 3, and uses quite a bit of functionality in it (XML-RPC,  
> skinning, and some hackery to build up the REST support). Getting  
> away from the current publisher won't be trivial.

Then why do it? (ditto)

> We couldn't make do with just the new publisher, but perhaps the new  
> publisher + extensions will be simpler to comprehend still than the  
> old publisher.

I encourage you to study zope.publisher.paste.  I content that the  
publisher itself isn't that complicated.

> Note a few issues I had with the current publisher that might be  
> resolved in a new one:
>
> REST support cannot be distinguished reliably

I don't know what you mean by REST support. Could you briefly explain?

> with a header (like I believe XML-RPC can). It needs to work in a  
> proper browser. This means that REST support (low-level handlers for  
> GET, POST, PUT, etc) needs to be mixed in an interesting way with  
> the browser publisher. For PUT and DELETE this is not a problem, as  
> the browser normally doesn't issue them. With POST and GET, the only  
> distinction you can make from normal browser behavior is actually  
> what URL is being used in the first place, and thus what view you're  
> trying to get. In Grok's REST support I used a different skin to  
> indicate this different kind of view. This required some hackery to  
> support. It'd be nice if there were a cleaner way to pass GET and  
> POST requests into separate handlers based on, say, the skin you're  
> in. Perhaps Stephan Richter's rest package can do this; I haven't  
> examined it yet.

I don't really know what you are talking about.  I'll make a few notes  
that may be responsive, or simply argumentative. :)

- I hate skins.  They cause me only pain. They cause lots of subtle  
complexity. I acknowledge that they provide some benefits but I don't  
think the benefits are worth the cost.  IMO, the problems that they  
solve can be solved in less magic ways. Note that skins are not  
intrinsic to the publisher.  They were imposed on the publisher in the  
complicated request-factory dance.

- I think the current mechanisms for dealing with XML-RPC was a  
mistake. It's waaay too complicated. The whole request-factory dance  
contributed to the complexity of the publisher configuration.

- I fixed a bug in the publisher over the weekend that may have  
contributed to your REST woes (whatever those are:).  I've been doing  
lots of ajax (sans xml :) lately.  Following standard ExtJS practice,  
I've been making requests as form data and getting JSON responses. I  
wanted my requests to be JSON too and found that POST requests were  
swallowing non-form/non-multi-part bodies. Now in zope.publisher. 
3.5.1, these bodies are left intact.

- To deal with AJAX requests, rather than changing the request setup  
logic to introduce a new request factory , I've localized the  
specialized behavior in descriptors (exposed as decorators).  I use  
simple views that return pages that load JS applications. These JS  
applications make ajax calls to sub-urls of the original pages.  In my  
view classes, I then have simple ajax methods that take and return  
Python data.  Something like:

     @zc.extjs.application.jsonpage
     def add(self, value):
         if not isinstance(value, int):
             return dict(error="The value must be an integer!")
         return self.do_add(value)

I'll be releasing this code with documentation in a week or two. I  
don't bring this up to plug my work, but merely to point out that  
there are simpler and. more importantly, more localized ways to  
approach protocols layered on HTTP.

> Security proxies: this topic may not be directly publisher related,

It's not.

> but maybe it is.

Nope. :)

> Somewhere quite low in the request handling of Zope 3 a security  
> proxy is introduced around the objects being traversed.

This is done by the default Zope 3 publication object,  
zope.app.publication.zopepublication.ZopePublication.

> Grok doesn't want security proxies, so rips them off again in a  
> custom publisher. It'd be nice if there were a hook point that would  
> enable us not introducing this proxy in the first place.


There always has been, as I've mentioned many times. The same hook  
point allows use of Zope 3 without ZODB.  These policies are provided  
by the publication object.  It's a shame that the convoluted setup  
system made this so inaccessible.

My hope is that zope.publisher.paste makes it *much* easier to use a  
different publications and thus a different set of policies. (I plan  
to update zc.zope3recipes to provide an instance recipe pased on paste.)

I'm 99% sure that most or all of the simplicity to want has already  
been there for a long time or is there now with the simpler setup  
framework in zope.publisher.paste.

Note that my proposal wasn't to simplify the publisher.  I think it  
already is pretty simple and I hope the recent zope.publisher.paste  
work exposes the simplicity by stripping off layers of overly complex  
setup.  My proposal was simply to extract some core functionality in a  
way that greatly reduces dependencies.  I don't expect this stripped  
down version to be of interest to most of the zope community.  I do  
think it might be useful to people with much more limited needs.  I  
didn't the proposal because I thought it would be of general interest,  
but to ask permission.  I want to do this refactoring, to avoid a (yet  
another) fork of the publisher. I should have made this clearer.

Jim

--
Jim Fulton
Zope Corporation