Reducing dependencies of zope.publisher
I'd like reduce the barrier of entry for using the Zope 3 zope publisher. Right now, it has quite a few dependencies setuptools, zope.component, zope.event, zope.exceptions, zope.i18n, zope.interface, zope.location, zope.proxy, zope.security, zope.deprecation, and zope.deferredimport. This seems a bit extreme. I can get rid of zope.deprecation and zope.deferredimport directly. I'd like to get rid of more. I'm thinking of creating a separate package that contains much of the guts of zope.publisher, but without most or all of the existing dependencies. I'd then modify zope.publisher to import code from this other package, adding the bits that exist now that cause the dependencies. This new package would a number of things, including skin support, xmlrpc support, mapply, interface declarations and various view base classes that now live in zope.publisher. Thoughts? Objections? Jim -- Jim Fulton Zope Corporation
On Mar 21, 2008, at 11:09 AM, Benji York wrote:
Jim Fulton wrote:
This new package would a number of things [...]
This new package would do what to a number of things?
Gaaah. Sorry. It would omit them. :) Jim -- Jim Fulton Zope Corporation
Hi Jim. What does this mean? Will the new package be a drop in replacement for what we have without some direction on what to do with the other bit or will it break our applications. A working publisher is pretty essential to a functioning zope app. Does this mean the end of deprecation warnings for the publisher? Is this not useful? Many thanks. Regards, David Jim Fulton wrote:
I'd like reduce the barrier of entry for using the Zope 3 zope publisher. Right now, it has quite a few dependencies setuptools, zope.component, zope.event, zope.exceptions, zope.i18n, zope.interface, zope.location, zope.proxy, zope.security, zope.deprecation, and zope.deferredimport. This seems a bit extreme. I can get rid of zope.deprecation and zope.deferredimport directly. I'd like to get rid of more.
I'm thinking of creating a separate package that contains much of the guts of zope.publisher, but without most or all of the existing dependencies. I'd then modify zope.publisher to import code from this other package, adding the bits that exist now that cause the dependencies.
This new package would a number of things, including skin support, xmlrpc support, mapply, interface declarations and various view base classes that now live in zope.publisher.
Thoughts? Objections?
Jim
-- Jim Fulton Zope Corporation
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists -http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
On Mar 21, 2008, at 12:10 PM, David Pratt wrote:
Hi Jim. What does this mean?
It wouldn't have any visible effect on zope.publisher. Applications that use zope.publisher wouldn't see a change. There would be an alternative to zope.publisher that has far fewer dependencies and fewer features. zope.publisher would build on this lower-level thing. The low-level package would be well suited to people who want to build small web applications with Python. As people have more requirements, they might graduate from the smaller framework to zope.publisher. Jim -- Jim Fulton Zope Corporation
Hi Jim. OK great. Many thanks for elaborating. This will be progressive. I had been considering an application use case without a zodb. Is this the scenario that the basic publisher would facilitate? Regards, David Jim Fulton wrote:
On Mar 21, 2008, at 12:10 PM, David Pratt wrote:
Hi Jim. What does this mean?
It wouldn't have any visible effect on zope.publisher. Applications that use zope.publisher wouldn't see a change.
There would be an alternative to zope.publisher that has far fewer dependencies and fewer features. zope.publisher would build on this lower-level thing. The low-level package would be well suited to people who want to build small web applications with Python. As people have more requirements, they might graduate from the smaller framework to zope.publisher.
Jim
-- Jim Fulton Zope Corporation
On Mar 21, 2008, at 6:54 PM, David Pratt wrote:
Hi Jim. OK great. Many thanks for elaborating. This will be progressive. I had been considering an application use case without a zodb. Is this the scenario that the basic publisher would facilitate?
No-more so than the existing publisher. I just want to make the publisher useful for small applications that don't need the component architecture and other facilities that zope.publisher depends on for mostly minor reasons. Jim -- Jim Fulton Zope Corporation
Jim Fulton wrote:
On Mar 21, 2008, at 6:54 PM, David Pratt wrote:
Hi Jim. OK great. Many thanks for elaborating. This will be progressive. I had been considering an application use case without a zodb. Is this the scenario that the basic publisher would facilitate?
No-more so than the existing publisher. I just want to make the publisher useful for small applications that don't need the component architecture and other facilities that zope.publisher depends on for mostly minor reasons.
This is of course a good goal in itself, though I'm curious how it compares to the Repoze publisher and its "obob" machinery? Cheers, Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book
On Mar 23, 2008, at 2:27 PM, Martin Aspeli wrote:
Jim Fulton wrote:
On Mar 21, 2008, at 6:54 PM, David Pratt wrote:
Hi Jim. OK great. Many thanks for elaborating. This will be progressive. I had been considering an application use case without a zodb. Is this the scenario that the basic publisher would facilitate? No-more so than the existing publisher. I just want to make the publisher useful for small applications that don't need the component architecture and other facilities that zope.publisher depends on for mostly minor reasons.
This is of course a good goal in itself, though I'm curious how it compares to the Repoze publisher and its "obob" machinery?
Me too. I have a hard time figuring out what repoze is and I don't necessarily want to go swimming to find out. :) Is the repose publisher described somewhere? I would like to create a 21st-century Bobo and this minimal publisher is part of that plan. If I create a new Bobo, I want it's requirements to be minimal. I don't want to depend on something as big as repoze seems to be. Jim -- Jim Fulton Zope Corporation
Jim Fulton wrote:
On Mar 23, 2008, at 2:27 PM, Martin Aspeli wrote:
Jim Fulton wrote:
On Mar 21, 2008, at 6:54 PM, David Pratt wrote:
Hi Jim. OK great. Many thanks for elaborating. This will be progressive. I had been considering an application use case without a zodb. Is this the scenario that the basic publisher would facilitate? No-more so than the existing publisher. I just want to make the publisher useful for small applications that don't need the component architecture and other facilities that zope.publisher depends on for mostly minor reasons. This is of course a good goal in itself, though I'm curious how it compares to the Repoze publisher and its "obob" machinery?
Me too. I have a hard time figuring out what repoze is and I don't necessarily want to go swimming to find out. :) Is the repose publisher described somewhere?
I would like to create a 21st-century Bobo and this minimal publisher is part of that plan. If I create a new Bobo, I want it's requirements to be minimal. I don't want to depend on something as big as repoze seems to be.
I think Repoze is a collection of many very small parts, rather than a very big whole. To me (i.e. to Plone), it's a way to deploy Zope 2 applications (Plone) in a WSGI way with pipelines and filters and all the rest of it. Repoze is also splitting out some of the useful bits of a monolithic Zope into re-usable middleware, including virtual hosting, transaction co-ordination, retry, and profiling. I suspect that the reason the Repoze guys ended up rewriting the Zope 2 publisher is that it was too monolithic to support deployment as a WSGI application. Anyway, I'm sure Chris and Tres can give better answers. ;) You may find http://blog.repoze.org/sixmonthsofrepoze.html a short and useful overview. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book
On Mar 23, 2008, at 3:02 PM, Martin Aspeli wrote:
Jim Fulton wrote:
On Mar 23, 2008, at 2:27 PM, Martin Aspeli wrote:
Jim Fulton wrote:
On Mar 21, 2008, at 6:54 PM, David Pratt wrote:
Hi Jim. OK great. Many thanks for elaborating. This will be progressive. I had been considering an application use case without a zodb. Is this the scenario that the basic publisher would facilitate? No-more so than the existing publisher. I just want to make the publisher useful for small applications that don't need the component architecture and other facilities that zope.publisher depends on for mostly minor reasons. This is of course a good goal in itself, though I'm curious how it compares to the Repoze publisher and its "obob" machinery? Me too. I have a hard time figuring out what repoze is and I don't necessarily want to go swimming to find out. :) Is the repose publisher described somewhere? I would like to create a 21st-century Bobo and this minimal publisher is part of that plan. If I create a new Bobo, I want it's requirements to be minimal. I don't want to depend on something as big as repoze seems to be.
I think Repoze is a collection of many very small parts, rather than a very big whole. To me (i.e. to Plone), it's a way to deploy Zope 2 applications (Plone) in a WSGI way with pipelines and filters and all the rest of it. Repoze is also splitting out some of the useful bits of a monolithic Zope into re-usable middleware, including virtual hosting, transaction co-ordination, retry, and profiling.
OK, I get that from the repoze about page. That doesn't explain why it should be relevant to my proposal to create a stripped-down Zope 3 publisher.
I suspect that the reason the Repoze guys ended up rewriting the Zope 2 publisher is that it was too monolithic to support deployment as a WSGI application.
Well, that's not my problem, on 2 counts. 1) I'm not using the Zope2 publisher. :) 2) The Zope3 publisher works just fine with WSGI and Paste. I suppose Tres and Chris didn't reuse the Zope 3 publisher for this because .... actually, I have no idea why they didn't. I'm pretty sure the Zope 3 publisher has the necessary hooks to allow someone to implement Zope 2 publisher semantics. Jim -- Jim Fulton Zope Corporation
On Mar 23, 2008, at 3:12 PM, Jim Fulton wrote:
No-more so than the existing publisher. I just want to make the publisher useful for small applications that don't need the component architecture and other facilities that zope.publisher depends on for mostly minor reasons. This is of course a good goal in itself, though I'm curious how it compares to the Repoze publisher and its "obob" machinery? Me too. I have a hard time figuring out what repoze is and I don't necessarily want to go swimming to find out. :) Is the repose publisher described somewhere?
The (relatively) nontechnical overview is at http://repoze.org/about.html The "repoze.obob" "publisher" (it's actually more of a meta-publisher) is described here: http://svn.repoze.org/repoze.obob/trunk/doc/ intro.txt ... it's mostly a "traffic cop" which delegates most of the hard work to plugins. repoze.zope2's "Zope2ObobHelper" (in http://svn.repoze.org/repoze.zope2/trunk/repoze/zope2/z2bob.py) is one such helper. More generally, Repoze is a brand that mainly implies "WSGI components inspired by Zope, which can be used outside of Zope". The individual components (see http://svn.repoze.org) are quite small and can be installed and used independently from each other, e.g.: Middleware ---------- repoze.tm -- WSGI middleware that uses the Zope transaction machinery to supply a transaction policy in a WSGI pipeline repoze.errorlog -- WSGI middleware that serves a purpose similar to that of the Zope 2 "error_log" object. repoze.who -- WSGI middleware that serves a purpose similar to that of Zope 2's PAS. repoze.profile -- WSGI middleware that can be used to profile applications. repoze.retry -- WSGI middleware that can retry a request based on a configurable policy (eg. a ConflictError) repoze.browserid -- WSGI middleware that serves a purpose similar to that of Zope 2's "browser_id_manager". repoze.decsec -- WSGI middleware that serves as a framework for defining and enforcing a declarative security policy. repoze.vhm -- WSGI middleware that serves a purpose similar to that of Zope 2's virtual host monster. None of these middleware components depend on each other and none of the configuration of any component depends on exposing the component architecture in any form to the end-user. Applications ------------ repoze.obob -- a WSGI application that implements a bobo-like publisher repoze.zope2 -- an obob plugin that implements Zope 2 publisher semantics repoze.kiss -- an obob plugin that implements filesystem serving of content repoze.grok -- an application which allows easy serving of Grok from a paste pipeline. repoze.plone -- a package which includes plone and which depends on repoze.zope2 repoze.django -- a WSGI application that allows you to run Django under a Paste pipeline repoze.trac -- a WSGI application that allows you to run Trac under a Paste pipeline Misc ---- repoze.recipe.egg -- a zc.buildout recipe which does the same thing as zc.recipe.egg but installs dependent scripts One point of all this stuff is mostly to be able to use Zope-like technologies in unfamilar environments (like Django, Pylons, etc) and carry knowledge across these platforms as necessary during consulting jobs. Relieving Zope of some of its duties (like error handling) allows us to trivially use middleware produced by people who aren't Zope people that is better suited for the task.
I would like to create a 21st-century Bobo and this minimal publisher is part of that plan. If I create a new Bobo, I want it's requirements to be minimal. I don't want to depend on something as big as repoze seems to be.
repoze.obob's setup.py says it depends on PasteScript and WSGIUtils. This is unfortunately a lie. It depends on only the stdlib.
I think Repoze is a collection of many very small parts, rather than a very big whole. To me (i.e. to Plone), it's a way to deploy Zope 2 applications (Plone) in a WSGI way with pipelines and filters and all the rest of it. Repoze is also splitting out some of the useful bits of a monolithic Zope into re-usable middleware, including virtual hosting, transaction co-ordination, retry, and profiling.
OK, I get that from the repoze about page. That doesn't explain why it should be relevant to my proposal to create a stripped-down Zope 3 publisher.
It isn't, except that you *might* want to consider making the new z3 publisher a repoze.obob plugin. Or not. The basics are there, but it's not very compelling unless you want to let folks change publishing semantics arbitrarily by supplying alternate plugin implementations.
I suspect that the reason the Repoze guys ended up rewriting the Zope 2 publisher is that it was too monolithic to support deployment as a WSGI application.
Well, that's not my problem, on 2 counts. 1) I'm not using the Zope2 publisher. :) 2) The Zope3 publisher works just fine with WSGI and Paste. I suppose Tres and Chris didn't reuse the Zope 3 publisher for this because .... actually, I have no idea why they didn't. I'm pretty sure the Zope 3 publisher has the necessary hooks to allow someone to implement Zope 2 publisher semantics.
I'm pretty sure that no one actually knew the Zope 2 publisher semantics until we reimplemented the Z2 publisher. The semantics are unreasonable. But at this point, given that we've now reimplemented it, and we understand them, it might be possible to use the Z3 publisher to emulate them. It'd be hard to tell without actually doing it and there are no plans to do so at the moment, though. For repoze.grok we did indeed just use the current WSGI app support in zope.app.wsgi, so repoze.grok is much simpler than repoze.zope2. Some observations we had while dealing with the current Z3 publisher: although the current Z3 publisher has a lot of policy knobs, they seem to be in the wrong places. For example, IIRC, in repoze.grok we had a lot of difficulty trying to figure out how to make the publisher *not* catch errors (we want to let WSGI middleware handle errors). We did eventually figure that out ("we" meaning Tres anyway), but we never did figure how to prevent the publisher from handling its own transaction management, nor did we figure out how to publish a different root object than the ZODB root, nor did we figure out how to prevent the publisher from doing its own conflict retry management. I'm sure all of these things are possible, but it's currently a bit complicated. Additionally, because zope.publisher currently depends on other pieces of Zope3 which themselves have dependencies, and so on, it's effectively pretty big (~18 eggs currently). This was not optimal. I'm a strong +1 on a simpler, smaller Z3 publisher. The only relationship this *might* have to Repoze is that we'd like the publisher to do a lot less, so we can handle some of its current duties (e.g. transaction management, error handling, retry) in middleware. Optimally without making the middleware depend on the component architecture to do so. Of course, we'd (or at least I'd) be willing to help in the effort. - C
On Fri, Mar 21, 2008 at 3:46 PM, Jim Fulton <jim@zope.com> wrote:
Thoughts? Objections?
I've caught the repoze bug, and if this makes a Repoze.zope3 easier to do/happier, then I'm all for it. In fact, I wouldn't mind if repoze became the official zope3 story. It would definitely help towards the goal of making Zope2 a Zope3 product. :) -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64
On Mar 21, 2008, at 12:15 PM, Lennart Regebro wrote:
On Fri, Mar 21, 2008 at 3:46 PM, Jim Fulton <jim@zope.com> wrote:
Thoughts? Objections?
I've caught the repoze bug, and if this makes a Repoze.zope3 easier to do/happier, then I'm all for it.
I suspect the work I did a couple of weeks ago adding paste support to zope.publisher would go a long way. That was released in zope.publisher 3.5.0. Jim -- Jim Fulton Zope Corporation
On Mar 21, 2008, at 12:55 PM, Jim Fulton wrote:
On Mar 21, 2008, at 12:15 PM, Lennart Regebro wrote:
On Fri, Mar 21, 2008 at 3:46 PM, Jim Fulton <jim@zope.com> wrote:
Thoughts? Objections?
I've caught the repoze bug, and if this makes a Repoze.zope3 easier to do/happier, then I'm all for it.
I suspect the work I did a couple of weeks ago adding paste support to zope.publisher would go a long way. That was released in zope.publisher 3.5.0.
Then again, since this makes it easy to use zope3 in paste, I don't know what repose adds. Jim -- Jim Fulton Zope Corporation
Jim Fulton wrote:
On Mar 21, 2008, at 12:55 PM, Jim Fulton wrote:
On Mar 21, 2008, at 12:15 PM, Lennart Regebro wrote:
On Fri, Mar 21, 2008 at 3:46 PM, Jim Fulton <jim@zope.com> wrote:
Thoughts? Objections? I've caught the repoze bug, and if this makes a Repoze.zope3 easier to do/happier, then I'm all for it. I suspect the work I did a couple of weeks ago adding paste support to zope.publisher would go a long way. That was released in zope.publisher 3.5.0.
Then again, since this makes it easy to use zope3 in paste, I don't know what repose adds.
It splits virtual hosting, transaction management, error logging and retry out as WSGI middleware. I'm not sure how much of that is shared between Zope 2 and Zope 3, of course. Really, I see Repoze as being mainly about patterns and packaging. I'm sure Chris and Tres would like to maintain as little software as possible. One thing that sucks right now for the repoze.zope2 story is that Zope 2 isn't "officially" packaged in an egg-friendly form so the Repoze guys have to repackage it. It'd be great to be able to get Zope 2 released in a way that can at least optionally be installable as eggs. Cheers, Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book
--On 21. März 2008 19:20:46 +0000 Martin Aspeli <optilude@gmx.net> wrote:
One thing that sucks right now for the repoze.zope2 story is that Zope 2 isn't "officially" packaged in an egg-friendly form so the Repoze guys have to repackage it. It'd be great to be able to get Zope 2 released in a way that can at least optionally be installable as eggs.
I think this should be a major goal for Zope 2.12. We possibly will have a Zope-(2 oriented) sprint in summer in Germany. If there are volunteers and interest I could add this to the possible sprint topics. Feedback welcome. Andreas
Andreas Jung wrote:
--On 21. März 2008 19:20:46 +0000 Martin Aspeli <optilude@gmx.net> wrote:
One thing that sucks right now for the repoze.zope2 story is that Zope 2 isn't "officially" packaged in an egg-friendly form so the Repoze guys have to repackage it. It'd be great to be able to get Zope 2 released in a way that can at least optionally be installable as eggs.
I think this should be a major goal for Zope 2.12. We possibly will have a Zope-(2 oriented) sprint in summer in Germany. If there are volunteers and interest I could add this to the possible sprint topics. Feedback welcome.
Awesome! Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book
On Mar 21, 2008, at 3:35 PM, Andreas Jung wrote:
--On 21. März 2008 19:20:46 +0000 Martin Aspeli <optilude@gmx.net> wrote:
One thing that sucks right now for the repoze.zope2 story is that Zope 2 isn't "officially" packaged in an egg-friendly form so the Repoze guys have to repackage it. It'd be great to be able to get Zope 2 released in a way that can at least optionally be installable as eggs.
I think this should be a major goal for Zope 2.12. We possibly will have a Zope-(2 oriented) sprint in summer in Germany. If there are volunteers and interest I could add this to the possible sprint topics. Feedback welcome.
Personally, I'd be happy if Zope2 was shipped as a single egg almost exactly like we ship it here: http://dist.repoze.org/zopelib-2.10.5.0.tar.gz The one thing I'd like to see is ZODB (and "transaction") removed from it. This is currently a little hard given that some C extensions in Zope depend on ZODB headers. Fixing that would likely be less than a day's worth of work for a sufficiently motivated person. I'm -1 on breaking it out into many eggs like Z3 is broken out now unless there's some super-compelling reason to do so for some limited number of packages (e.g. I can see StructuredText coming out). - C
Jim Fulton wrote: [snip]
Thoughts? Objections?
A simpler publisher has been on my wish-list for a long time now. I'm a bit worried though that a publisher born from the current Zope 3 publisher with the goal to build up enough support for the Zope 3 publisher to make use of the code will not in fact be simpler. The current Zope 3 publisher supports quite a few pluggability points (though not necessarily always in the most convenient places), and has a lot of interacting components, which makes it rather hard to comprehend sometimes. You seem to be confident you can create this simplicity while retaining enough pluggability to build the Zope 3 publisher back up on it, though, correct? I'd be interested in using such a publisher as the foundation for the publisher in Grok. Of course Grok does currently use the current Zope 3 publisher in Zope 3, and uses quite a bit of functionality in it (XML-RPC, skinning, and some hackery to build up the REST support). Getting away from the current publisher won't be trivial. We couldn't make do with just the new publisher, but perhaps the new publisher + extensions will be simpler to comprehend still than the old publisher. Note a few issues I had with the current publisher that might be resolved in a new one: REST support cannot be distinguished reliably with a header (like I believe XML-RPC can). It needs to work in a proper browser. This means that REST support (low-level handlers for GET, POST, PUT, etc) needs to be mixed in an interesting way with the browser publisher. For PUT and DELETE this is not a problem, as the browser normally doesn't issue them. With POST and GET, the only distinction you can make from normal browser behavior is actually what URL is being used in the first place, and thus what view you're trying to get. In Grok's REST support I used a different skin to indicate this different kind of view. This required some hackery to support. It'd be nice if there were a cleaner way to pass GET and POST requests into separate handlers based on, say, the skin you're in. Perhaps Stephan Richter's rest package can do this; I haven't examined it yet. Security proxies: this topic may not be directly publisher related, but maybe it is. Somewhere quite low in the request handling of Zope 3 a security proxy is introduced around the objects being traversed. Grok doesn't want security proxies, so rips them off again in a custom publisher. It'd be nice if there were a hook point that would enable us not introducing this proxy in the first place. Regards, Martijn
On Mar 25, 2008, at 5:25 AM, Martijn Faassen wrote:
Jim Fulton wrote: [snip]
Thoughts? Objections?
A simpler publisher has been on my wish-list for a long time now.
I'm a bit worried though that a publisher born from the current Zope 3 publisher with the goal to build up enough support for the Zope 3 publisher to make use of the code will not in fact be simpler. The current Zope 3 publisher supports quite a few pluggability points (though not necessarily always in the most convenient places), and has a lot of interacting components, which makes it rather hard to comprehend sometimes.
I suspect you are confusing, as the repoze folks seem to have, the publisher, with the way that the Zope 3 server set up code wires it up. When we originally set up zope.server at the dawn of Zope 3, we inadvertently created a gordian knot of interacting components. When we added WSGI and Twisted support, we just made this worse by adding additional threads. The basic publisher framework is pretty straightforward. There is a generic publisher and a publication component that takes care of customizing the publisher for a particular application. The publication interface, zope.publisher.interfaces.IPublication, defines the available hooks. The recent addition of zope.publisher.paste makes selection of a custom publisher much simpler -- as well as supplying paste support. Unfortunately, I don't think anyone has paid attention to this. :( If all you want is more control over how the publisher works, I think the current publisher simplified via the new paste support should meet your needs. It's possible some folks need more hooks, but then we should focus on evolving the publication API, zope.publisher.interfaces.IPublication.
You seem to be confident you can create this simplicity while retaining enough pluggability to build the Zope 3 publisher back up on it, though, correct?
Yup. In fact, all I'm planning to do is to move most of the existing publisher code to a separate package sans interface declarations, xmlrpc, and a few other things that many applications don't need. I intend to keep the current publisher+publication framework. I don't expect to rewrite any code, other than tests, or change any semantics. In fact, I'd like migration from the leaner pubisher to the full publisher to be as easy as in import change.
I'd be interested in using such a publisher as the foundation for the publisher in Grok.
Why? (But read to the end before answering. :)
Of course Grok does currently use the current Zope 3 publisher in Zope 3, and uses quite a bit of functionality in it (XML-RPC, skinning, and some hackery to build up the REST support). Getting away from the current publisher won't be trivial.
Then why do it? (ditto)
We couldn't make do with just the new publisher, but perhaps the new publisher + extensions will be simpler to comprehend still than the old publisher.
I encourage you to study zope.publisher.paste. I content that the publisher itself isn't that complicated.
Note a few issues I had with the current publisher that might be resolved in a new one:
REST support cannot be distinguished reliably
I don't know what you mean by REST support. Could you briefly explain?
with a header (like I believe XML-RPC can). It needs to work in a proper browser. This means that REST support (low-level handlers for GET, POST, PUT, etc) needs to be mixed in an interesting way with the browser publisher. For PUT and DELETE this is not a problem, as the browser normally doesn't issue them. With POST and GET, the only distinction you can make from normal browser behavior is actually what URL is being used in the first place, and thus what view you're trying to get. In Grok's REST support I used a different skin to indicate this different kind of view. This required some hackery to support. It'd be nice if there were a cleaner way to pass GET and POST requests into separate handlers based on, say, the skin you're in. Perhaps Stephan Richter's rest package can do this; I haven't examined it yet.
I don't really know what you are talking about. I'll make a few notes that may be responsive, or simply argumentative. :) - I hate skins. They cause me only pain. They cause lots of subtle complexity. I acknowledge that they provide some benefits but I don't think the benefits are worth the cost. IMO, the problems that they solve can be solved in less magic ways. Note that skins are not intrinsic to the publisher. They were imposed on the publisher in the complicated request-factory dance. - I think the current mechanisms for dealing with XML-RPC was a mistake. It's waaay too complicated. The whole request-factory dance contributed to the complexity of the publisher configuration. - I fixed a bug in the publisher over the weekend that may have contributed to your REST woes (whatever those are:). I've been doing lots of ajax (sans xml :) lately. Following standard ExtJS practice, I've been making requests as form data and getting JSON responses. I wanted my requests to be JSON too and found that POST requests were swallowing non-form/non-multi-part bodies. Now in zope.publisher. 3.5.1, these bodies are left intact. - To deal with AJAX requests, rather than changing the request setup logic to introduce a new request factory , I've localized the specialized behavior in descriptors (exposed as decorators). I use simple views that return pages that load JS applications. These JS applications make ajax calls to sub-urls of the original pages. In my view classes, I then have simple ajax methods that take and return Python data. Something like: @zc.extjs.application.jsonpage def add(self, value): if not isinstance(value, int): return dict(error="The value must be an integer!") return self.do_add(value) I'll be releasing this code with documentation in a week or two. I don't bring this up to plug my work, but merely to point out that there are simpler and. more importantly, more localized ways to approach protocols layered on HTTP.
Security proxies: this topic may not be directly publisher related,
It's not.
but maybe it is.
Nope. :)
Somewhere quite low in the request handling of Zope 3 a security proxy is introduced around the objects being traversed.
This is done by the default Zope 3 publication object, zope.app.publication.zopepublication.ZopePublication.
Grok doesn't want security proxies, so rips them off again in a custom publisher. It'd be nice if there were a hook point that would enable us not introducing this proxy in the first place.
There always has been, as I've mentioned many times. The same hook point allows use of Zope 3 without ZODB. These policies are provided by the publication object. It's a shame that the convoluted setup system made this so inaccessible. My hope is that zope.publisher.paste makes it *much* easier to use a different publications and thus a different set of policies. (I plan to update zc.zope3recipes to provide an instance recipe pased on paste.) I'm 99% sure that most or all of the simplicity to want has already been there for a long time or is there now with the simpler setup framework in zope.publisher.paste. Note that my proposal wasn't to simplify the publisher. I think it already is pretty simple and I hope the recent zope.publisher.paste work exposes the simplicity by stripping off layers of overly complex setup. My proposal was simply to extract some core functionality in a way that greatly reduces dependencies. I don't expect this stripped down version to be of interest to most of the zope community. I do think it might be useful to people with much more limited needs. I didn't the proposal because I thought it would be of general interest, but to ask permission. I want to do this refactoring, to avoid a (yet another) fork of the publisher. I should have made this clearer. Jim -- Jim Fulton Zope Corporation
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jim Fulton wrote:
On Mar 25, 2008, at 5:25 AM, Martijn Faassen wrote:
Jim Fulton wrote: [snip]
Thoughts? Objections? A simpler publisher has been on my wish-list for a long time now.
I'm a bit worried though that a publisher born from the current Zope 3 publisher with the goal to build up enough support for the Zope 3 publisher to make use of the code will not in fact be simpler. The current Zope 3 publisher supports quite a few pluggability points (though not necessarily always in the most convenient places), and has a lot of interacting components, which makes it rather hard to comprehend sometimes.
I suspect you are confusing, as the repoze folks seem to have, the publisher, with the way that the Zope 3 server set up code wires it up.
I spent a couple of days trying to work out how those bits were interacting, back before we re-implemented a bobo-like thing as 'repoze.obob'. Its intro doc lays out our goals: - ------------------------------------------- repoze.obob Overview This package provides a paste-configurable version of the classic 'bobo' publisher, which maps request URLs to objects via graph traversal. It is desgined to be the endpoint ("application") in a WSGI filter / application pipeline. The publisher is responsible for: - converting the WSGI environment into a request object and its corresponding resopnse object; - selecting the "root" object of the graph for a given request / URL; - traversing from that root object along the "edges" defined by the URL path elements to find the "published object"; - invoking the published object, mapping any request parameters onto its arguments; - mapping response headers and body, along with the result from calling the published object, into appropriate WSGI output. The publisher is *not* responsible for the following tasks, which belong to WSGI "middleware" components: - setting up any transaction; - committing or aborting a transaction; - retrying failed / conflicting requests; - converting Python exceptions into HTTP response codes. - ---------------------------------------- In particular, we wanted to move all those excluded responsibilities out into middleware, so that they could be shared across multiple WSGI applications, including those which were not Zope-specific at all. We've been successfull at that goal: the Pylons and TurbooGears folks are working on integrating repoze.tm into their stacks, for instance. repoze.obob puts all the policy for publishing into a set of plugins passed in at construction (typically configured via Paste configuration): - Initializing anything at startup - Finding the root object - Handling the policy-driven bits of the publishing traversal, calling the published object, and converting the result. This part is largely like what the publication object does, except the "find the root object" bits, which we broke out. We have a re-implementation of the classic Zope2 publisher machinery (repoze.zope2.z2bob) which suppy these policy implementations. repoze.kiss, for instance, works by just replacing the "root finder" part with version which creates a proxy object for a filesystem directory: the "Zope2ObobHelper" then works fine against that alternate root.
When we originally set up zope.server at the dawn of Zope 3, we inadvertently created a gordian knot of interacting components. When we added WSGI and Twisted support, we just made this worse by adding additional threads.
The publication factory dance is completely inscrutable.
The basic publisher framework is pretty straightforward. There is a generic publisher and a publication component that takes care of customizing the publisher for a particular application. The publication interface, zope.publisher.interfaces.IPublication, defines the available hooks. The recent addition of zope.publisher.paste makes selection of a custom publisher much simpler -- as well as supplying paste support. Unfortunately, I don't think anyone has paid attention to this. :(
I haven't looked much into it. I would be happy to have a simpler way to set up the Zope3 publisher, free of the policy stuff embedded in the existing publication objects.
If all you want is more control over how the publisher works, I think the current publisher simplified via the new paste support should meet your needs. It's possible some folks need more hooks, but then we should focus on evolving the publication API, zope.publisher.interfaces.IPublication.
Can we implement a publication which defers error handling / transaction handling / retry handling to middleware? The 'repoze.grok' package, for instance, works around the error handling bit by adding a WSIG "ingress" filter which stuffs a key into the WSGI environment ('wsgi.handleErrors') to get the Zope3 WSGI application to defer error handling. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFH6Vd2+gerLs4ltQ4RAlX9AJ94LIc9s125p6cGsitBXDoN6g0bfwCff7uZ B8ztzcxpF4hw+qBHXxtxZeE= =AASc -----END PGP SIGNATURE-----
I didn't know about zope.publisher.paste, thank you. I've spent a little bit of time trying to grok zope.publisher.publish and the IPublication API, and indeed it seems there is almost a one- for-one mapping between Zope's "publication" and repoze.obob's "helper". It would almost certainly be possible to use a slimmed-down zope.publisher to do what repoze.obob/repoze.zope2 does now. But it would mean: - Implementing something Zope2-specific that implemented IPublication. - Implementing a new request type that handled traversal in a Zope2- ish way, that didn't support retry (as that's meant to be handled upstream). But essentially, if somebody was motivated, the code in Zope2ObobHelper at http://svn.repoze.org/repoze.zope2/trunk/repoze/zope2/z2bob.py could mostly be cut-n-pasted to allow the Z3 publisher to serve up a Zope 2 app. Two nits: - I don't like the fact that zope.publisher.publish uses the "setResult" API of the request to handle the result returned by the published object.. why wouldn't it just return the result? Maybe this dance was cargo-culted over from Zope2. - I don't like that the request handles traversal. This also smells like it was cargo-culted from Z2. Why shouldn't the publication itself do traversal? Traversal is extremely policy-laden, why break the code that implements that policy up so that you need "matching" request and publication objects? - C On Mar 25, 2008, at 3:50 PM, Tres Seaver wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Jim Fulton wrote:
On Mar 25, 2008, at 5:25 AM, Martijn Faassen wrote:
Jim Fulton wrote: [snip]
Thoughts? Objections? A simpler publisher has been on my wish-list for a long time now.
I'm a bit worried though that a publisher born from the current Zope 3 publisher with the goal to build up enough support for the Zope 3 publisher to make use of the code will not in fact be simpler. The current Zope 3 publisher supports quite a few pluggability points (though not necessarily always in the most convenient places), and has a lot of interacting components, which makes it rather hard to comprehend sometimes.
I suspect you are confusing, as the repoze folks seem to have, the publisher, with the way that the Zope 3 server set up code wires it up.
I spent a couple of days trying to work out how those bits were interacting, back before we re-implemented a bobo-like thing as 'repoze.obob'. Its intro doc lays out our goals:
- ------------------------------------------- repoze.obob Overview
This package provides a paste-configurable version of the classic 'bobo' publisher, which maps request URLs to objects via graph traversal. It is desgined to be the endpoint ("application") in a WSGI filter / application pipeline.
The publisher is responsible for:
- converting the WSGI environment into a request object and its corresponding resopnse object;
- selecting the "root" object of the graph for a given request / URL;
- traversing from that root object along the "edges" defined by the URL path elements to find the "published object";
- invoking the published object, mapping any request parameters onto its arguments;
- mapping response headers and body, along with the result from calling the published object, into appropriate WSGI output.
The publisher is *not* responsible for the following tasks, which belong to WSGI "middleware" components:
- setting up any transaction;
- committing or aborting a transaction;
- retrying failed / conflicting requests;
- converting Python exceptions into HTTP response codes.
- ----------------------------------------
In particular, we wanted to move all those excluded responsibilities out into middleware, so that they could be shared across multiple WSGI applications, including those which were not Zope-specific at all. We've been successfull at that goal: the Pylons and TurbooGears folks are working on integrating repoze.tm into their stacks, for instance.
repoze.obob puts all the policy for publishing into a set of plugins passed in at construction (typically configured via Paste configuration):
- Initializing anything at startup
- Finding the root object
- Handling the policy-driven bits of the publishing traversal, calling the published object, and converting the result. This part is largely like what the publication object does, except the "find the root object" bits, which we broke out.
We have a re-implementation of the classic Zope2 publisher machinery (repoze.zope2.z2bob) which suppy these policy implementations. repoze.kiss, for instance, works by just replacing the "root finder" part with version which creates a proxy object for a filesystem directory: the "Zope2ObobHelper" then works fine against that alternate root.
When we originally set up zope.server at the dawn of Zope 3, we inadvertently created a gordian knot of interacting components. When we added WSGI and Twisted support, we just made this worse by adding additional threads.
The publication factory dance is completely inscrutable.
The basic publisher framework is pretty straightforward. There is a generic publisher and a publication component that takes care of customizing the publisher for a particular application. The publication interface, zope.publisher.interfaces.IPublication, defines the available hooks. The recent addition of zope.publisher.paste makes selection of a custom publisher much simpler -- as well as supplying paste support. Unfortunately, I don't think anyone has paid attention to this. :(
I haven't looked much into it. I would be happy to have a simpler way to set up the Zope3 publisher, free of the policy stuff embedded in the existing publication objects.
If all you want is more control over how the publisher works, I think the current publisher simplified via the new paste support should meet your needs. It's possible some folks need more hooks, but then we should focus on evolving the publication API, zope.publisher.interfaces.IPublication.
Can we implement a publication which defers error handling / transaction handling / retry handling to middleware? The 'repoze.grok' package, for instance, works around the error handling bit by adding a WSIG "ingress" filter which stuffs a key into the WSGI environment ('wsgi.handleErrors') to get the Zope3 WSGI application to defer error handling.
Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFH6Vd2+gerLs4ltQ4RAlX9AJ94LIc9s125p6cGsitBXDoN6g0bfwCff7uZ B8ztzcxpF4hw+qBHXxtxZeE= =AASc -----END PGP SIGNATURE-----
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
On Mar 25, 2008, at 4:41 PM, Chris McDonough wrote: ...
Two nits:
- I don't like the fact that zope.publisher.publish uses the "setResult" API of the request to handle the result returned by the published object..
You mean the response....
why wouldn't it just return the result?
Because the response object is likely to do additional processing. It is likely to set response headers at a minimum, but it may do much more.
Maybe this dance was cargo-culted over from Zope2.
I don't think so.
- I don't like that the request handles traversal.
It doesn't do much more than break the path into segments. All of the actual traversal is performed by the publication. The publication doesn't have to consume one segment at a time. It can consume the entire path on the first traversal if it wants to. It can also do traversal during initial object computation.
This also smells like it was cargo-culted from Z2.
I don't care for this characterization. When we did the Zope 3 publisher, we borrowed heavily from the Zope 2 one. The Zope 2 publisher is pretty heavily battle tested. I like the fact that traversal is done a step at a time. It allows me to distribute the traversal logic. If an application wants to traverse on the URL as a whole, they can do that in the initial object selection.
Why shouldn't the publication itself do traversal?
It does,
Traversal is extremely policy-laden, why break the code that implements that policy up so that you need "matching" request and publication objects?
You don't. The request part of traversal is very general and only protocol specific. To my knowledge, no one has ever written a custom request to implement custom application policy. AFAIK, the publication API is sufficient. BTW, I don't claim that zope.publisher is flawless. I'd be happy to see some discussion of it's strengths and weaknesses for the purpose of making it better. I think my biggest regret is changing the request and response APIs as much from those in Zope 2 as we did. I think it would be interesting to look at how difficult it would be to reunify the Zope 2 and Zope 3 request and response APIs. It might also be interesting to look at a broader unification as Ian Bicking has made some movement toward. Jim -- Jim Fulton Zope Corporation
On Mar 25, 2008, at 5:05 PM, Jim Fulton wrote:
On Mar 25, 2008, at 4:41 PM, Chris McDonough wrote: ...
Two nits:
- I don't like the fact that zope.publisher.publish uses the "setResult" API of the request to handle the result returned by the published object..
You mean the response....
Yes, thank you.
why wouldn't it just return the result?
Because the response object is likely to do additional processing. It is likely to set response headers at a minimum, but it may do much more.
Why shouldn't the publication be responsible for setting response headers based on the result and doing that additional processing?
- I don't like that the request handles traversal.
It doesn't do much more than break the path into segments. All of the actual traversal is performed by the publication.
That's not how zope.publisher:publish reads, though: request.processInputs() publication.beforeTraversal(request) obj = publication.getApplication(request) obj = request.traverse(obj) publication.afterTraversal(request, obj) result = publication.callObject(request, obj) The Z3 request.traverse implementation happens to delegate back to the publication using method calls against its "self.publication" attr. Isn't this kind of convoluted? Maybe it should just be publication.traverse(request, obj) to match the other API methods of a publication? e.g.: request.processInputs() publication.beforeTraversal(request) obj = publication.getApplication(request) obj = publication.traverse(request, obj) publication.afterTraversal(request, obj) result = publication.callObject(request, obj)
This also smells like it was cargo-culted from Z2.
I don't care for this characterization.
Sorry, no harm intended.
When we did the Zope 3 publisher, we borrowed heavily from the Zope 2 one. The Zope 2 publisher is pretty heavily battle tested. I like the fact that traversal is done a step at a time. It allows me to distribute the traversal logic. If an application wants to traverse on the URL as a whole, they can do that in the initial object selection.
Sure, no argument there, I'm really just arguing against publish.py delegating to a request's traverse method.
Why shouldn't the publication itself do traversal?
It does,
Traversal is extremely policy-laden, why break the code that implements that policy up so that you need "matching" request and publication objects?
You don't. The request part of traversal is very general and only protocol specific. To my knowledge, no one has ever written a custom request to implement custom application policy. AFAIK, the publication API is sufficient.
I agree that the IPublication API is correct. I'd just prefer that the request not be responsible for performing top-level traversal dispatch which then must call *back in* to the publication.
BTW, I don't claim that zope.publisher is flawless. I'd be happy to see some discussion of it's strengths and weaknesses for the purpose of making it better. I think my biggest regret is changing the request and response APIs as much from those in Zope 2 as we did. I think it would be interesting to look at how difficult it would be to reunify the Zope 2 and Zope 3 request and response APIs. It might also be interesting to look at a broader unification as Ian Bicking has made some movement toward.
Yup. - C
On Tue, Mar 25, 2008 at 9:19 AM, Jim Fulton <jim@zope.com> wrote:
On Mar 25, 2008, at 5:25 AM, Martijn Faassen wrote:
Security proxies: this topic may not be directly publisher related,
It's not.
but maybe it is.
Nope. :)
Somewhere quite low in the request handling of Zope 3 a security proxy is introduced around the objects being traversed.
This is done by the default Zope 3 publication object, zope.app.publication.zopepublication.ZopePublication.
i've deployed apps with a custom zopepublication in part to bypass security proxies, and its unfortunately not the only place that injects security proxies. the more difficult injection point to avoid is the one in zope/app/pagetemplate/engine.zopeTraverser
Grok doesn't want security proxies, so rips them off again in a custom publisher. It'd be nice if there were a hook point that would enable us not introducing this proxy in the first place.
There always has been, as I've mentioned many times. The same hook point allows use of Zope 3 without ZODB. These policies are provided by the publication object. It's a shame that the convoluted setup system made this so inaccessible.
My hope is that zope.publisher.paste makes it *much* easier to use a different publications and thus a different set of policies. (I plan to update zc.zope3recipes to provide an instance recipe pased on paste.)
I'm 99% sure that most or all of the simplicity to want has already been there for a long time or is there now with the simpler setup framework in zope.publisher.paste.
Note that my proposal wasn't to simplify the publisher. I think it already is pretty simple and I hope the recent zope.publisher.paste work exposes the simplicity by stripping off layers of overly complex setup. My proposal was simply to extract some core functionality in a way that greatly reduces dependencies. I don't expect this stripped down version to be of interest to most of the zope community. I do think it might be useful to people with much more limited needs. I didn't the proposal because I thought it would be of general interest, but to ask permission. I want to do this refactoring, to avoid a (yet another) fork of the publisher. I should have made this clearer.
this all sounds really nice. i've been doing rdb apps with zope with custom publications.. but getting the initial setup right for wsgi involved some dead chickens to reimplementing the the wsgi application setup that had zodb bits setup and event firing hardcoded. looking over the new zope.publisher.paste code, it looks like it also pushes the responsibility for application setup to the publication factory.. ie. zcml loading, application startup events. +1 cheers, kapil
participants (10)
-
Andreas Jung -
Benji York -
Chris McDonough -
David Pratt -
Jim Fulton -
Kapil Thangavelu -
Lennart Regebro -
Martijn Faassen -
Martin Aspeli -
Tres Seaver