On Sat, Mar 19, 2011 at 13:07, Lennart Regebro <regebro@gmail.com> wrote:

On Thu, Mar 17, 2011 at 22:08, Jim Fulton <jim@zope.com> wrote:

...

On Thu, Mar 17, 2011 at 2:57 PM, Lennart Regebro <regebro@gmail.com> wrote:

...

I'm still in Atlanta, and Arc Riley asked for a Zope person to possibly mentor some zope.* project for Python Software Foundation this year. They probably want to get more of the Zope Toolkit ported to Python 3. I forwarded the roadmap to him, so anyone who wants to mentor, that would be great.

I hope that isn't what they want. :)

His stated goal for GSOC was to bring people into the community. I can't think of many better ways to turn someone off than to ask them to port something from Py2 to Py3.

Well, that's what he said. But in any case it might be a good idea if Zope people joined the PSF GSoC project to vote and comment on any zope related applications if those show up. I will, even though I refuse to be a mentor, since I suck at it.

Ah, actually, this year you put together "teams". So in this case we need a Zope team. So if nobody shows interest we don't have a team, and we won't do anything. //Lennart

On Saturday, March 19, 2011, Lennart Regebro wrote:

Getting ZCA/ZTK to run on PyPy is probably easier, and actually more useful. Maybe someone would want to mentor that?

While I would mentor someone wanting to do such a project, I would be much more interested in seeing a working WebOb to zope.publisher bridge. I know Jim(?) has done some initial work on that. I think it would make an interesting PSF project, since it encourages more reusability across the Python Web dev community. So I'll sign up as a Zope team member. Regards, Stephan -- Entrepreneur and Software Geek Google me. "Zope Stephan Richter"

On Sun, Mar 20, 2011 at 01:06, Stephan Richter <srichter@cosmos.phy.tufts.edu> wrote:

On Saturday, March 19, 2011, Lennart Regebro wrote:

...

Getting ZCA/ZTK to run on PyPy is probably easier, and actually more useful. Maybe someone would want to mentor that?

While I would mentor someone wanting to do such a project, I would be much more interested in seeing a working WebOb to zope.publisher bridge. I know Jim(?) has done some initial work on that. I think it would make an interesting PSF project, since it encourages more reusability across the Python Web dev community.

So I'll sign up as a Zope team member.

Cool. But it turns out we need three to make a team (see https://spreadsheets.google.com/viewform?formkey=dHh3WFNGYzkyWWE0ZjM1eFFoUUV...), and we only really have one. :-) I guess I could take a bullet for the team too, but that still makes only two. Maybe next year. :-) //Lennart

On Mon, Mar 21, 2011 at 13:28, Stephan Richter <srichter@cosmos.phy.tufts.edu> wrote:

On Monday, March 21, 2011, Lennart Regebro wrote:

...

...

So I'll sign up as a Zope team member.

Cool. But it turns out we need three to make a team (see https://spreadsheets.google.com/viewform?formkey=dHh3WFNGYzkyWWE0ZjM1eFFoUU VGWmc6MQ), and we only really have one. :-) I guess I could take a bullet for the team too, but that still makes only two. Maybe next year. :-)

Jim said he would be willing to mentor someone for the right project. That would make us three.

Ah, OK. I'll sign up then.

On 03/20/2011 02:31 PM, Jim Fulton wrote:

On Sat, Mar 19, 2011 at 5:02 PM, Lennart Regebro<regebro@gmail.com> wrote:

...

Getting ZCA/ZTK to run on PyPy is probably easier, and actually more useful. Maybe someone would want to mentor that?

This is another porting project. If I was a student, I wouldn't find it very interesting to port some code I hadn't written and probably don't understand from one platform to another. (My earlier comment on py2k->py3k porting wasn't meant as a dig against py3k bit rather to say that I didn't think porting projects would be very interesting.)

That's true, we had bad experience with the Jython project a few years ago. It depends on the students though; I think we had better experience with a project to move Zope 2 forward to a newer version of Python 2.x (if I recall that correctly). But writing new code is a lot more fun. But if you're going to do porting, PyPy is a lot more tempting as a target for me. Regards, Martijn

On Sun, Mar 20, 2011 at 9:49 AM, Martin Aspeli <optilude+lists@gmail.com> wrote:

On 20 March 2011 13:46, Jim Fulton <jim@zope.com> wrote:

...

I think we ought to come up with a much cleaner way of defining default configuration. (Pyramid does this by passing default values in adapter calls, but I think we can do a lot better than that.)  I'd like to see us come up with a "pythonic" way to wire components up that can be overridden through registration (through zcml or otherwise).  Ideally, the mechanism shouldn't "feel" like "configuration" but like "programming".

You mean like

http://pypi.python.org/pypi/grokcore.component http://pypi.python.org/pypi/grokcore.view http://pypi.python.org/pypi/grokcore.viewlet http://pypi.python.org/pypi/grokcore.security

?

Hm, it's been a while since I've looked at grok. Some notes: - The mechanism I'm thinking of should not require *any* ZCML. - The mechanism shouldn't require something to "grok"/analyze the code. The mechanism should be explicit. This is implied by "pythonic". I remember Grok being more implicit than skimming the links above suggest. Perhaps Grok has has become more explicit than I remember. - I'd prefer not to rely on subclassing, but I'm not dead set against it. - Whataever we come up with has to work with Python 3, so unfortunately, we can't use the syntactic trick of having a call in a class like:: grokcore.component.implements(IContentProvider) The effort should certainly include an analysis of approaches like grok. Maybe the mechanism should have the effect of enabling tools like grok to be implemented more cleanly. Note that the move from "advice-based" syntax to class decorators provides a good opportunity to revisit how some of this works based on experience using the ztk. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton

On 3/20/11 16:00 , Hanno Schlichting wrote:

On Sun, Mar 20, 2011 at 3:28 PM, Jim Fulton<jim@zope.com> wrote:

...

- The mechanism shouldn't require something to "grok"/analyze the code. The mechanism should be explicit. This is implied by "pythonic". I remember Grok being more implicit than skimming the links above suggest. Perhaps Grok has has become more explicit than I remember.

Both Grok and Pyramid (or martian and venusian really) do a scan of the code to find the registration hints.

Pyramid only does so if you tell it to do so by using config.scan(). You are not obliged to do that, and I have several pyramid projects which do not do any scanning. Not doing scanning has the advantage of making configuration more explicit, and it speeds application startup immensely. Scanning is needed if you want to mix configuration and code in one place. That may not be the right model for everyone. Wichert.

On 3/20/11 16:12 PM, Wichert Akkerman wrote:

...

...

Both Grok and Pyramid (or martian and venusian really) do a scan of the code to find the registration hints.

Pyramid only does so if you tell it to do so by using config.scan(). You are not obliged to do that, and I have several pyramid projects which do not do any scanning. Not doing scanning has the advantage of making configuration more explicit, and it speeds application startup immensely.

Just to get this clear for me: if you're not scanning, the information left by the class decorators would be "inert"? So, you'd have to do the registrations "yourself", right?

Let me try to argue this better. Downsides of scanning are:

- it scans your tests, which can result in unexpected behaviour you may not expect (at least for venusian, not sure if this is true for martian).

Martian skips tests by default. You can tell it, at least to a certain extend, what to scan and what not to scan.

- you may have some draft files in your tree that are not ready for use and never referenced anywhere, but a scan will still process them.

This is true.

- scanning can take a long time, making application (re)start slow for non-trivial projects

At what point is an application not trivial anymore? In applications I build so far, startup time has not been an issue at all. But maybe my applications are still on the trivial-end of the spectrum ;)

- problems in the scanning process tend to be very hard debug. If a view is not processed during scanning figuring out why can be painful, and there are little to no tools to help you. This is especially true for more complex scanning environments such as the plone/dexterity/z3cform stack; as an example I spent over an hour yesterday trying to figure out why a form was not picked up while other views in the same python file worked fine.

I think this can be true. In my experience not relying on implicitly or "guessed configuration parameters helps a little here. What in this specific example was the reason for the view not being picked up? regards, jw

On 3/21/11 10:30 AM, Wichert Akkerman wrote:

...

...

- you may have some draft files in your tree that are not ready for use and never referenced anywhere, but a scan will still process them.

This is true.

I ran into this with .html.py files generated by Chameleon as well. My Zope startup has lots of these:

/Users/wichert/Library/eggs/zope.configuration-3.6.0-py2.6.egg/zope/configuration/config.py:605: UserWarning: File 'sessions.pt.pyc' has an unrecognized extension in directory '/Users/wichert/Work/syslab/euphorie/Develop/trunk/buildout/src/Euphorie/euphorie/client/templates'

I think this particular warning is not due to the scanning process itself, but due to the way Grok tries to associate templates to views. It is still true though that the .pt.py files generated by chameleon will be scanned anyway. regards, jw

On Sun, Mar 20, 2011 at 11:00 AM, Hanno Schlichting <hanno@hannosch.eu> wrote:

On Sun, Mar 20, 2011 at 3:28 PM, Jim Fulton <jim@zope.com> wrote:

...

- The mechanism shouldn't require something to "grok"/analyze the  code.  The mechanism should be explicit. This is implied by  "pythonic".  I remember Grok being more implicit than skimming the  links above suggest. Perhaps Grok has has become more explicit than  I remember.

Both Grok and Pyramid (or martian and venusian really) do a scan of the code to find the registration hints.

I think you cannot avoid this, if you want to support an explicit configuration phase. Otherwise the first import of a module could occur at any point at runtime and have a configuration side-effect like registering a new view. Personally I find the venusian approach pretty simple and explicit enough.

I disagree. First, the notion that you'd import at run time is pretty odd, unless you count start-up in "runtime". Second, well, really, I'm not ready to give up on a straightforward definition of wiring that doesn't rely on module scanning.

...

Note that the move from "advice-based" syntax to class decorators provides a good opportunity to revisit how some of this works based on experience using the ztk.

Taking one of the examples of grokcore.component, I think there's a lot that can be made simpler:

import grokcore.component from zope.i18n.interfaces import ITranslationDomain

class HelloWorldTranslationDomain(grokcore.component.GlobalUtility):    grokcore.component.implements(ITranslationDomain)    grokcore.component.name('helloworld')

Based on my Pyramid exposure, I'd write this as something like:

from something import utility from zope.i18n.interfaces import ITranslationDomain

@utility(ITranslationDomain, name='helloworld') class HelloWorldTranslationDomain(object):    pass

This does mean hardcoding default values into the registration calls. I'm not sure I see the value of avoiding this, as long as there's a way to unregister this utility and re-register the class with other values.

Yup. Also, If there's an existing mechanism that does what I want and I'm just ignorant of, I'd be happy to learn about it. If such a thing exists or can be cobbled together from existing ideas, I'd like to elevate it either as part of the ZCA or as a best-practice tool that stands independent of any particular application framework. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton

On Sun, Mar 20, 2011 at 4:29 PM, Jim Fulton <jim@zope.com> wrote:

I disagree.  First, the notion that you'd import at run time is pretty odd, unless you count start-up in "runtime".

Right, sorry. I'm used to writing add-ons for an application. In this environment my code isn't in charge of the startup process, but will be called sometime in the middle of it. Some of your code might be imported early, some very late. In this sort of unpredictable environment, scanning comes in handy. Once some of your code is called, you can ensure to scan and register all of it.

Second, well, really, I'm not ready to give up on a straightforward definition of wiring that doesn't rely on module scanning.

Sure. I didn't mean to exclude this. Pyramid allows you to do a very explicit configuration without any scanning. If you write an application and have full control over all its parts, this works. Things get complicated, once you reuse libraries which in turn have other library dependencies with configuration actions. Ignore the naming for the moment. You could call these application components, arguing that a library shouldn't have any hard configuration requirements. Currently we have an explicit approach via providing a configure.zcml in each library, which causes all relevant parts to be imported and hooked up. One library can include the ZCML of another library or a subpackage of its own. Either each library provides a similar central place, which imports all its modules with configuration actions, or you use the scanning approach to avoid the explicit mechanism.

Also, If there's an existing mechanism that does what I want and I'm just ignorant of, I'd be happy to learn about it.  If such a thing exists or can be cobbled together from existing ideas, I'd like to elevate it either as part of the ZCA or as a best-practice tool that stands independent of any particular application framework.

I think venusian, the venusian actions and the configuration machinery of Pyramid come pretty close to this. Personally I think we could extract these and roll them back into the ZTK. Maybe there's something in Grok that comes close to this. I've just not been able to distinguish the "Grok the web framework" code with its convention over configuration idea from some basic explicit configuration approach. Hanno

Hi there, On 03/20/2011 05:47 PM, Hanno Schlichting wrote:

Maybe there's something in Grok that comes close to this. I've just not been able to distinguish the "Grok the web framework" code with its convention over configuration idea from some basic explicit configuration approach.

I don't understand. Venusian took its ideas from Martian, and Martian's not a web framework and never has been. http://pypi.python.org/pypi/martian Surely you've heard of this library before? Regards, Martijn

Hi, On 20 March 2011 15:00, Hanno Schlichting <hanno@hannosch.eu> wrote:

On Sun, Mar 20, 2011 at 3:28 PM, Jim Fulton <jim@zope.com> wrote:

...

- The mechanism shouldn't require something to "grok"/analyze the  code.  The mechanism should be explicit. This is implied by  "pythonic".  I remember Grok being more implicit than skimming the  links above suggest. Perhaps Grok has has become more explicit than  I remember.

Both Grok and Pyramid (or martian and venusian really) do a scan of the code to find the registration hints.

I think you cannot avoid this, if you want to support an explicit configuration phase. Otherwise the first import of a module could occur at any point at runtime and have a configuration side-effect like registering a new view. Personally I find the venusian approach pretty simple and explicit enough.

...

Note that the move from "advice-based" syntax to class decorators provides a good opportunity to revisit how some of this works based on experience using the ztk.

Taking one of the examples of grokcore.component, I think there's a lot that can be made simpler:

import grokcore.component from zope.i18n.interfaces import ITranslationDomain

class HelloWorldTranslationDomain(grokcore.component.GlobalUtility):    grokcore.component.implements(ITranslationDomain)    grokcore.component.name('helloworld')

Based on my Pyramid exposure, I'd write this as something like:

from something import utility from zope.i18n.interfaces import ITranslationDomain

@utility(ITranslationDomain, name='helloworld') class HelloWorldTranslationDomain(object):    pass

This does mean hardcoding default values into the registration calls. I'm not sure I see the value of avoiding this, as long as there's a way to unregister this utility and re-register the class with other values.

I agree - this is nicer. In Grok's defence, class decorators didn't exist when martian was conceived. :) I do wonder whether it's nicer *though*, though, to justify yet more proliferation of configuration syntax. I appreciate that in Python 3 the in-class advice (which was pioneered by zope.component/zope.interface, don't forget) may not work properly, so we may not have any choice eventually. But saving two lines of code in a hypothetical example doesn't necessarily outweigh the confusion that comes from having multiple ways to do things, and grokcore.* configuration is in use in the wild outside of Grok. Martin

Hi there, On 03/20/2011 04:00 PM, Hanno Schlichting wrote:

Taking one of the examples of grokcore.component, I think there's a lot that can be made simpler:

import grokcore.component from zope.i18n.interfaces import ITranslationDomain

class HelloWorldTranslationDomain(grokcore.component.GlobalUtility): grokcore.component.implements(ITranslationDomain) grokcore.component.name('helloworld')

Based on my Pyramid exposure, I'd write this as something like:

from something import utility from zope.i18n.interfaces import ITranslationDomain

@utility(ITranslationDomain, name='helloworld') class HelloWorldTranslationDomain(object): pass

It's interesting to consider inheritance rules here. What if you subclass from HelloWorldTranslationDomain? What happens to name and the interface that the utility is provided under? I'm not saying I know the right rules for inheritance in all cases, or that grokcore.component is sane here, but I know in some cases having directive values inherit is pretty neat, and in some cases it isn't. I imagine registration can always be explicit, however. Note also that if we're simply talking spelling, this makes grok a bit shorter and is the way Grok code typically looks: import grokcore.component as grok from zope.i18n.interfaces import ITranslationDomain class HelloWorldTranslationDomain(grok.GlobalUtility): grok.implements(ITranslationDomain) grok.name('helloworld') Regards, Martijn

Hi, Not sure where to 'hook into' the discussion thread, so I'll just start here: On 3/20/11 15:28 PM, Jim Fulton wrote:

Hm, it's been a while since I've looked at grok. Some notes:

- The mechanism I'm thinking of should not require *any* ZCML.

Do you mean "without having a configure.zcml trigger the scanning (a.k.a. grokking) process"? Or do you mean, "a mechanism that doesn't use configuration actions from zope.configuration"? If there's a different way to trigger the scanning process, that would be mostly fine for Grok. There's a couple of things actually configured thru ZCML files in some of the grokcore.* packages. I think they could be rewritten if necessary. ZCML files are as far as I can see not essential to Grok - they are surely not essential to martian (the scanning/grokking library used).

- The mechanism shouldn't require something to "grok"/analyze the code. The mechanism should be explicit. This is implied by "pythonic". I remember Grok being more implicit than skimming the links above suggest. Perhaps Grok has has become more explicit than I remember.

I have the feeling the "explicit versus implicit" part of the discussion has been somewhat mixed with the "to scan or not to scan" part of the discussion. Nothing in the martian library dictates "implicitness" as far as I can see. How Grok uses martian though, does have implicit, conventions-over-configuration design choices. Actually, these "conventions-over-configuration" choices are regularly discussed within the Grok community - some people (including myself) would like to see Grok not doing any "guessing" of configuration parameters during the grokking-phase of the application at all. Others disagree. As an example, there's the "megrok.strictrequire" package that, when included in a grok-based application, will raise grok errors when there are view components without an explicitly set permission requirement (where Grok normally would have used a conventional value).

- I'd prefer not to rely on subclassing, but I'm not dead set against it.

For most components Grok relies on subclassing indeed. Note that registering global utilities, for example, can be done using module-level directives too, like can be done for adapters and subscriptions.

- Whataever we come up with has to work with Python 3, so unfortunately, we can't use the syntactic trick of having a call in a class like::

grokcore.component.implements(IContentProvider)

Just to be sure: this is what is called "advice based syntax" or "in-class advice" right? Grok people call this a "directive". Anyway, as apparently this wouldn't work in Python 3 anymore, Grok should come up with an alternate spelling. Actually, people have already suggested to start using class decorators instead of in-class directives. Personally, I do not see an essential cosmetic difference between using a class decorator or a in-class directive. Like was said earlier in the thread: Grok had to use directives since there were no class decorators at that time. And what I have seen of Pyramid and venusian, the grok directives do mostly what the class decorators do: they leave information on the class in one way or another, that later is picked up in the scanning/grokking phase and used for registrations. Grok uses "grokkers" for that - "meta"-components that know how they can use the information left by the directives for making registrations (thru the zope.configuration configuration actions mechanism, which as a result plays nice with existing "pure zcml" registrations) What I know of venusian/Pyramid, is that the class decorators leave callbacks that will do the registrations in the scanning phase. Right?

The effort should certainly include an analysis of approaches like grok. Maybe the mechanism should have the effect of enabling tools like grok to be implemented more cleanly.

I do not think the Grok project would be principally against this. regards, jw

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 03/20/2011 09:46 AM, Jim Fulton wrote:

Problem =======

ZTK projects use ZCML too much. Ideally, ZCML should only have to be used when we want to override something.

Solution sketch ===============

I think we ought to come up with a much cleaner way of defining default configuration. (Pyramid does this by passing default values in adapter calls, but I think we can do a lot better than that.)

I'm not confident that "better" is achievable for the classic "dependency injection" case (i.e., the default is the one you almost always want, except for unit testing). Typically, I define a utility interface *and* the default implementation in the module which mostly uses it. E.g.:: from zope.interface import Interface from zope.component import queryUtility class ISomePlugPoint(Interface): def __call__(foo, bar): """blah blah""" # Look, Ma! No decorator! def defaultImpl(foo, bar): # DTRT for the normal case def clientFunction(request): impl = queryUtility(ISomePlugPoint, default=defaultImpl) Note the absence extra declarations for 'defaultImpl', and of extra syntax of any kind. In order to maintain good test isolation, I usually avoid memoizing the lookup (e.g. as a module scope variable). unless profiling shows that the lookup ends up on a critical path. This pattern even works outside of testing: if you do the frameworky thing and document how to override the utility as a policy, it becomes a trivial integration point. For adapters, the example is a bit noisier, because the component registry wants to call the factory for you:: from zope.interface import Interface from zope.component import queryAdapter class IAdaptsTo(Interface): def someMethod(): """ blah, blah.""" class DefaultImpl(object): # Note that we require no decorator or advice def __init__(self, context): self.context = context def someMethod(self): return 'whatever' def anotherClient(context, request): adapted = queryAdapter(context, IAdaptsTo) if adapted is None: adapted = DefaultImpl(context) # now use it (Note that memoization isn't a practical optimization doesn't work for adapters.) If we added a 'default_factory' argument to 'queryAdapter', we wouldn't need the 'if' statement, which would make this example as compact as the utility version. Or we could add a 'queryAdapterFactory' API instead, and have 'queryAdapter' use it (what is one more function call between friends? ;) The one downside I can see is giving up on the sugar^Wexpressivity of calling the interface directly -- I guess we could propagate the 'default_factory' argument through to the '__call__' of interface. Note that I *wanted* some extra sugar at one point (doing utility lookup when no arguments were passed to Interface.__call__), but I haven't missed that convenience much since I went on a low sugar diet with BFG / pyramid.

I'd like to see us come up with a "pythonic" way to wire components up that can be overridden through registration (through zcml or otherwise). Ideally, the mechanism shouldn't "feel" like "configuration" but like "programming".

My example feels like programming to me: no ZCML, no decorators, and no advice needed up until the point you want to override the normal defaults. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk2GND0ACgkQ+gerLs4ltQ6ynwCeJ1u/Bk3u8LGxhgR1jk2CFQP3 ZrcAoIuZbFCh2dpB611jKvOlUx48alqo =lvhO -----END PGP SIGNATURE-----

On Tue, Apr 5, 2011 at 11:57, Lennart Regebro <regebro@gmail.com> wrote:

On Tue, Apr 5, 2011 at 07:56, Chris McDonough <chrism@plope.com> wrote:

...

Did this particular effort get to the place where there are students and mentors lined up to do ZTK porting?

No, it got to the pace where I'm supposed to set up a team page on a Wiki.

Done: http://wiki.zope.org/gsoc/SummerOfCode2011