Pluggins vs Application Definition
Some recent discussions on the distutils-sig mailing list have helped me to understand some issues related to the ways we extend the Zope application server. Traditionally, in Zope 2, you extended Zope by "dropping" product packages into a special "Products" package. This was very convenient in many ways, but doesn't always provide enough control or visibiity to what's going on. In Zope 3, we went with a more explicit installation mechanism, in which people had to explicitly cause a package's ZCML files to be loaded for it to be used. We added a mechanism to make this easier, by simply dropping a file into a special directory, package-includes, so an installer wouldn't have to fool with pointy brackets. I've noticed that at Zope Corporation, for our customer projects, we tend not to use package-includes. We prefer to explicitly include packages we use directly oin our application ZCML files. (As applications, may be composed of several layers, we may include things at multiple levels.) In thinking about this, I realized that there are two different users here with different concerns: - Application developers need to build an application. They will generally want fairly tight control over what goes into the application. For them, it's valuable to say in an explicit way what they want. - If the application is extensible, then application users will want to be able to extend the application by adding "pluggins". If application users are not technically sophisticated, or, more importantly, not technically interested, they peobably would prefer to just drop something into a special directory and be done with it. In summary, I think we need *both* approaches, as they serve different needs. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
Jim Fulton wrote:
Some recent discussions on the distutils-sig mailing list have helped me to understand some issues related to the ways we extend the Zope application server. Traditionally, in Zope 2, you extended Zope by "dropping" product packages into a special "Products" package. This was very convenient in many ways, but doesn't always provide enough control or visibiity to what's going on.
In Zope 3, we went with a more explicit installation mechanism, in which people had to explicitly cause a package's ZCML files to be loaded for it to be used. We added a mechanism to make this easier, by simply dropping a file into a special directory, package-includes, so an installer wouldn't have to fool with pointy brackets.
I've noticed that at Zope Corporation, for our customer projects, we tend not to use package-includes. We prefer to explicitly include packages we use directly oin our application ZCML files. (As applications, may be composed of several layers, we may include things at multiple levels.)
In thinking about this, I realized that there are two different users here with different concerns:
- Application developers need to build an application. They will generally want fairly tight control over what goes into the application. For them, it's valuable to say in an explicit way what they want.
- If the application is extensible, then application users will want to be able to extend the application by adding "pluggins". If application users are not technically sophisticated, or, more importantly, not technically interested, they peobably would prefer to just drop something into a special directory and be done with it.
In summary, I think we need *both* approaches, as they serve different needs.
I agree with all of the above, but fail to see what your implication is. I think we have all that now. Sure, we're not dropping Zope 3 plug-ins into a special directory but really anywhere in PYTHONPATH. Whether that additional freedom makes it easier is debatable, but I'd say we're still burned from Zope 2 wrt fixed drop-in locations. Frankly, I think something like "ez_install zope.reallycoolstuff" is even easier than manually dropping stuff. As for the package-includes slug, a simple ez_install-like tool could help there as well. I sketched it out once on a blog post: http://www.z3lab.org/sections/blogs/philipp-weitershausen/2006_01_11_mata-lo... Philipp
Philipp von Weitershausen wrote:
Jim Fulton wrote:
Some recent discussions on the distutils-sig mailing list have helped me to understand some issues related to the ways we extend the Zope application server. Traditionally, in Zope 2, you extended Zope by "dropping" product packages into a special "Products" package. This was very convenient in many ways, but doesn't always provide enough control or visibiity to what's going on.
In Zope 3, we went with a more explicit installation mechanism, in which people had to explicitly cause a package's ZCML files to be loaded for it to be used. We added a mechanism to make this easier, by simply dropping a file into a special directory, package-includes, so an installer wouldn't have to fool with pointy brackets.
I've noticed that at Zope Corporation, for our customer projects, we tend not to use package-includes. We prefer to explicitly include packages we use directly oin our application ZCML files. (As applications, may be composed of several layers, we may include things at multiple levels.)
In thinking about this, I realized that there are two different users here with different concerns:
- Application developers need to build an application. They will generally want fairly tight control over what goes into the application. For them, it's valuable to say in an explicit way what they want.
- If the application is extensible, then application users will want to be able to extend the application by adding "pluggins". If application users are not technically sophisticated, or, more importantly, not technically interested, they peobably would prefer to just drop something into a special directory and be done with it.
In summary, I think we need *both* approaches, as they serve different needs.
I agree with all of the above, but fail to see what your implication is.
Mostly thinking out loud and to lazy to bother blogging. :) Well, more than that. I think it's helpful to identify user's goals and problems to be solved rather than argue about solutions in isolation, as is commonly done.
I think we have all that now.
In Zope 3, we don't have anything like plugins. Zope 2 does and, thanks to *also* adopting ZCML *is* getting the best of both worlds, sort of. In neither case, IMO, to we really *separately* consider the separate needs of the application developer and the high-level user.
Sure, we're not dropping Zope 3 plug-ins into a special directory but really anywhere in PYTHONPATH. Whether that additional freedom makes it easier is debatable, but I'd say we're still burned from Zope 2 wrt fixed drop-in locations.
I'm not sure what you are saying here. Did you mean to say "burned from Zope 2"? (probably not)
Frankly, I think something like "ez_install zope.reallycoolstuff" is even easier than manually dropping stuff. As for the package-includes slug, a simple ez_install-like tool could help there as well. I sketched it out once on a blog post: http://www.z3lab.org/sections/blogs/philipp-weitershausen/2006_01_11_mata-lo...
I agree. In fact, on the distutils-sig mailing list, they are contemplating a plugin mechanism strikingly similar to Zope 2's for applications such as Chandler and I recommended a more explicit aproach, where plugins should be added under explicit user control. I think we need to consider both use cases and have stories (documentation, tools, etc.) for both and when designing these, we should consider the separate needs of the application developers and high-level users who don't need or *want* the same level of control. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
On Saturday 11 February 2006 16:50, Jim Fulton wrote:
- Application developers need to build an application. They will generally want fairly tight control over what goes into the application. For them, it's valuable to say in an explicit way what they want.
- If the application is extensible, then application users will want to be able to extend the application by adding "pluggins". If application users are not technically sophisticated, or, more importantly, not technically interested, they peobably would prefer to just drop something into a special directory and be done with it.
In summary, I think we need *both* approaches, as they serve different needs.
This is interesting. I agree with Philipp though that a simple install tool would be better than one magic location. I think the ZCML slugs are very cool and if we have a tool (as "make" does now) that does this one step, then we effectively have drop-in packages. BTW, I think that a tool is also better, because it would allow us to keep track of the installed packages and do dependency checking, package-db maintenance, etc. Just randomly thinking... Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
Stephan Richter wrote:
On Saturday 11 February 2006 16:50, Jim Fulton wrote:
- Application developers need to build an application. They will generally want fairly tight control over what goes into the application. For them, it's valuable to say in an explicit way what they want.
- If the application is extensible, then application users will want to be able to extend the application by adding "pluggins". If application users are not technically sophisticated, or, more importantly, not technically interested, they peobably would prefer to just drop something into a special directory and be done with it.
In summary, I think we need *both* approaches, as they serve different needs.
This is interesting. I agree with Philipp though that a simple install tool would be better than one magic location. I think the ZCML slugs are very cool and if we have a tool (as "make" does now) that does this one step, then we effectively have drop-in packages. BTW, I think that a tool is also better, because it would allow us to keep track of the installed packages and do dependency checking, package-db maintenance, etc. Just randomly thinking...
I also think an install script is a good way of facading the actual process. In fact for my pythonproducts product (allows one to use python packages as zope2 products without the Products directory) I was considering providing a py script to do this for zope2 products. Zope 2 developers see zcml slugs and say, "what, things have gotten harder going to zope3?". - Rocky -- Rocky Burt ServerZen Software -- http://www.serverzen.com ServerZen Hosting -- http://www.serverzenhosting.net News About The Server -- http://www.serverzen.net
Rocky Burt wrote:
Zope 2 developers see zcml slugs and say, "what, things have gotten harder going to zope3?".
This isn't a contradiction of what Rocky Burt said, but I feel the need to assert that "harder" doesn't necessarily mean "worse". It's also not necessarily "better" either. Now back to your regularly scheduled, non-truistic messages. -- Benji York Senior Software Engineer Zope Corporation
Stephan Richter wrote:
This is interesting. I agree with Philipp though that a simple install tool would be better than one magic location.
Indeed, but don't eggs already provide tools for this?
I think the ZCML slugs are very cool
I think they suck, sorry... it's one of the first things I hit with Zope 3 over Christmas :-(
and if we have a tool (as "make" does now)
Please god, not make...
that does this one step, then we effectively have drop-in packages.
No, drop in packages means you either use a package-manner like interface, like aptitude, a binary installer, or actually dropping a single file into a directory. cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
[aside... hmmm, crossposting, maybe time to merge zope-dev and zope3-dev? most stuff seems to be relevent to both nowadays] Jim Fulton wrote:
In Zope 3, we went with a more explicit installation mechanism, in which people had to explicitly cause a package's ZCML files to be loaded for it to be used. We added a mechanism to make this easier, by simply dropping a file into a special directory, package-includes, so an installer wouldn't have to fool with pointy brackets.
Personally, I hate package-includes. It creates a load of mess in a folder that is likely to need to have new tools written to look after it in the future :-/ I'd really like to just see lines getting includes in site.zcml. There is precedent for this, very good precedent in fact. Think of httpd.conf, and the module include lines at the top. I'm all for supporting having squillions of files all over the place, but I'd really like to see the default be one file, as with Apache. Also, I wish there was some way to have a .zcml that is in the _software_ home, not the package home, that inlcuded all the "standard" bits so that when I upgrade my zope version, I don't have to worry about juggling all those files and wondering which ones are out of date and which ones have "new" versions...
- If the application is extensible, then application users will want to be able to extend the application by adding "pluggins".
This also applies particularly to framework applications like CMF...
If application users are not technically sophisticated, or, more importantly, not technically interested, they peobably would prefer to just drop something into a special directory and be done with it.
Yup, /products, /plugins, etc, everything from Photoshop to Zope 2 has this concept ;-) cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Chris Withers wrote:
[aside... hmmm, crossposting, maybe time to merge zope-dev and zope3-dev? most stuff seems to be relevent to both nowadays]
+10 You know, I once had a proposal..... Uh, never mind :)
In Zope 3, we went with a more explicit installation mechanism, in which people had to explicitly cause a package's ZCML files to be loaded for it to be used. We added a mechanism to make this easier, by simply dropping a file into a special directory, package-includes, so an installer wouldn't have to fool with pointy brackets.
Personally, I hate package-includes. It creates a load of mess in a folder that is likely to need to have new tools written to look after it in the future :-/
I'd really like to just see lines getting includes in site.zcml.
We had that once. We got rid of it because it made deployment harder. With slugs you can just drop in an extra file without having to change one. That makes automated installation easy.
There is precedent for this, very good precedent in fact. Think of httpd.conf, and the module include lines at the top. I'm all for supporting having squillions of files all over the place, but I'd really like to see the default be one file, as with Apache.
Heh, that's actually a funny example because Debian has actually been using modular config files for quite some time now (not only for Apache). Especially module includes, module configurations and virtual sites all rest in individual config files. Small tools (a2enmod, a2ensite) help manage the files, though manual management isn't too hard either. So, seems like not only are we the first ones to come up with a setup like that, it's also used by a large
Also, I wish there was some way to have a .zcml that is in the _software_ home, not the package home, that inlcuded all the "standard" bits so that when I upgrade my zope version, I don't have to worry about juggling all those files and wondering which ones are out of date and which ones have "new" versions...
You should "know" which ones you want in your instance anyways. I rather have a very, very slim "standard" set of packages and make everything else optional than provide everything else by default. If the software home ZCML file (which essentially is zope.app/configure.zcml) includes too much, it's a pain to disable things you don't like. It's trivial with package-includes.
If application users are not technically sophisticated, or, more importantly, not technically interested, they peobably would prefer to just drop something into a special directory and be done with it.
Yup, /products, /plugins, etc, everything from Photoshop to Zope 2 has this concept ;-)
It's been giving us a lot of pain in Zope 2. I don't think anyone seriously wants to go back. We *can* have a standardized location (by convention), e.g. $INSTANCE_HOME/lib/python, but let's not make it a mandatory one. Philipp
On Wednesday 15 February 2006 07:20, Philipp von Weitershausen wrote:
Chris Withers wrote:
[aside... hmmm, crossposting, maybe time to merge zope-dev and zope3-dev? most stuff seems to be relevent to both nowadays]
+10
Okay, I hope this would be ignored, but -1. I can still ignore many E-mails on Zope-Dev. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
Philipp von Weitershausen wrote:
Chris Withers wrote:
[aside... hmmm, crossposting, maybe time to merge zope-dev and zope3-dev? most stuff seems to be relevent to both nowadays]
+10
You know, I once had a proposal..... Uh, never mind :)
Well, what needs to happen for it to be done? To be clear: I'm talking _only_ about merging the dev lists, _not_ the user lists. The users lists are still largely independent, but it seems like just about every post to the dev list now has a bearing on both Zope 2 and Zope 3, especially as they become closer and closers... It's going to happen at some stage, and I think now is that time. Stefan, at some point, will just have to get over himself ;-) Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Chris Withers wrote:
Philipp von Weitershausen wrote:
Chris Withers wrote:
[aside... hmmm, crossposting, maybe time to merge zope-dev and zope3-dev? most stuff seems to be relevent to both nowadays]
+10
You know, I once had a proposal..... Uh, never mind :)
Well, what needs to happen for it to be done?
We'd have to declare the zope3-dev list for obsolete and make people not send messages to it. We'd just have to define a date and time. All subsequent mail traffic would be handled by zope-dev.
To be clear: I'm talking _only_ about merging the dev lists, _not_ the user lists. The users lists are still largely independent, but it seems like just about every post to the dev list now has a bearing on both Zope 2 and Zope 3, especially as they become closer and closers...
Absolutely agreed (both on users lists and the common topics). As said, I'm +10. Philipp
Philipp von Weitershausen wrote:
We'd have to declare the zope3-dev list for obsolete and make people not send messages to it. We'd just have to define a date and time. All subsequent mail traffic would be handled by zope-dev.
Okay, who actually makes this call and how to we set a date? Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
On Feb 17, 2006, at 6:26 PM, Chris Withers wrote:
Philipp von Weitershausen wrote:
We'd have to declare the zope3-dev list for obsolete and make people not send messages to it. We'd just have to define a date and time. All subsequent mail traffic would be handled by zope-dev.
Okay, who actually makes this call and how to we set a date?
Only you and Philipp were excited about this. Not sure that constitutes a ringing endorsement. Maybe others will chime in now. FWIW, I'm -0. Gary
Gary Poster wrote:
On Feb 17, 2006, at 6:26 PM, Chris Withers wrote:
Philipp von Weitershausen wrote:
We'd have to declare the zope3-dev list for obsolete and make people not send messages to it. We'd just have to define a date and time. All subsequent mail traffic would be handled by zope-dev.
Okay, who actually makes this call and how to we set a date?
Only you and Philipp were excited about this. Not sure that constitutes a ringing endorsement. Maybe others will chime in now.
FWIW, I'm -0.
Same here. -- Benji York Senior Software Engineer Zope Corporation
if we're voting, I'm +0. -- Paul Winkler http://www.slinkp.com
Gary Poster wrote:
On Feb 17, 2006, at 6:26 PM, Chris Withers wrote:
Philipp von Weitershausen wrote:
We'd have to declare the zope3-dev list for obsolete and make people not send messages to it. We'd just have to define a date and time. All subsequent mail traffic would be handled by zope-dev.
Okay, who actually makes this call and how to we set a date?
Only you and Philipp were excited about this. Not sure that constitutes a ringing endorsement. Maybe others will chime in now.
I'm +10 too. I'd like to see this happen before the end of the year. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
On Friday 17 February 2006 23:54, Jim Fulton wrote:
Only you and Philipp were excited about this. Not sure that constitutes a ringing endorsement. Maybe others will chime in now.
I'm +10 too.
I'd like to see this happen before the end of the year.
-1. I only change my vote, if the technologies get much closer together. In other words when Zoep 2 uses Zope 3's security, publisher, Page Templates, RDB Adapters, etc. Currently I often just delete zope-dev mails, because some weird thing is discussed. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training
On 18 Feb 2006, at 13:08, Stephan Richter wrote:
On Friday 17 February 2006 23:54, Jim Fulton wrote:
Only you and Philipp were excited about this. Not sure that constitutes a ringing endorsement. Maybe others will chime in now.
I'm +10 too.
I'd like to see this happen before the end of the year.
-1. I only change my vote, if the technologies get much closer together. In other words when Zoep 2 uses Zope 3's security, publisher, Page Templates, RDB Adapters, etc.
Currently I often just delete zope-dev mails, because some weird thing is discussed.
So nothing will change for you then, Stephan. Your argument has just been fatally weakened ;) Anyhow, +1 from me as well. I don't mind deleting esoteric Zope 3 stuff *I* don't care about. Stephan's argument works both ways. jens
Jim Fulton wrote:
Only you and Philipp were excited about this. Not sure that constitutes a ringing endorsement. Maybe others will chime in now.
I'm +10 too.
I'd like to see this happen before the end of the year.
Well, given that the majority are +/-0 and with the exception of one or two misguided individuals who seem to want Zope 3 to live on its own forever, how do we set a date? Why does this need to be a big deal? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
On 2/16/06, Chris Withers <chris@simplistix.co.uk> wrote:
To be clear: I'm talking _only_ about merging the dev lists, _not_ the user lists. The users lists are still largely independent, but it seems like just about every post to the dev list now has a bearing on both Zope 2 and Zope 3, especially as they become closer and closers...
-1 -Fred -- Fred L. Drake, Jr. <fdrake at gmail.com> "There is no wealth but life." --John Ruskin
Fred Drake wrote:
On 2/16/06, Chris Withers <chris@simplistix.co.uk> wrote:
To be clear: I'm talking _only_ about merging the dev lists, _not_ the user lists. The users lists are still largely independent, but it seems like just about every post to the dev list now has a bearing on both Zope 2 and Zope 3, especially as they become closer and closers...
-1
Any particular reasons? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Chris Withers wrote:
maybe time to merge zope-dev and zope3-dev?
-1 They're still different enough that most traffic on one is not interesting to the majority of subscribers to the other. -- Benji York Senior Software Engineer Zope Corporation
Benji York wrote:
Chris Withers wrote:
maybe time to merge zope-dev and zope3-dev?
-1
They're still different enough that most traffic on one is not interesting to the majority of subscribers to the other.
Can you explain what lead you to this conclusion? It seems to be the complete opposite to me, to the point where I have them both filtering to the same folder and don't notice which list a mail is for anymore... Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
participants (10)
-
Benji York -
Chris Withers -
Fred Drake -
Gary Poster -
Jens Vagelpohl -
Jim Fulton -
Paul Winkler -
Philipp von Weitershausen -
Rocky Burt -
Stephan Richter