Why is TemplateDict so opaque? -and- Have you seen this problem?
[sorry folks, I'm tired, so this is halfway to a rant :-( ] I'm trying to do a work-around for the "namespace goes away after two levels of dtml method calls" problem with an XXXPythonMethod, and have now hit another wall as regards accessing the namespace enclosing the XXXPythonMethod. So I'm sitting here trying to figure out where this variable of mine is hidden, and I'm thinking I should be able to get there via the _ instance of TemplateDict. Well.... I'm not happy that I now know what TemplateDict is - I really wish I hadn't had to go there, but now that I'm there I have a much better understanding of why it is impossible to get information out of it; it has "do you have this key" and "get the value of this key" and absolutely nothing else. Is there a reason for this? I can think of *countless* times when the ability to just have a *glimpse* of my namespace would have saved hours. Is there a possibility that you could make the thing look more like a mapping object, with keys() and values() and items() and such? It'd sure save a ton of time. Some kind of tag to pretty-print the namespace would be great too. ... But then, actually, not having to deal with the namespace going away problem would have saved even more time. I'll describe it briefly to see if it rings any bells: Imagine a dtml method called methodA that looks like this: <!--#var standard_html_header--> Depth: <dtml-var "methodA(counter=counter+1)"> <!--#var standard_html_footer--> and you call it from a browser as methodA?counter:int=0 You'd expect one of these: a) zope locks up/crashes because of the recursion b) methodA bails out because someone detects recursion or exceeds a stack limit of some sort What you *get*, however, is the very suprising: Error type: KeyError Error value: standard_html_header And if you rip out the reference to standard_html_..., you'll get the equally baffling: Error type: NameError Error value: methodA In practice, you'll find that this disappearing act occurs at about the third level of expression evaluation. Ring any bells? It requires *horrible* workarounds when you run into it. Ok, thanks for putting up with this. Dave Parker Sr. Software Engineer GlobalCrossing
At 06:45 AM 12/16/99 -0600, Dave Parker wrote:
But then, actually, not having to deal with the namespace going away problem would have saved even more time. I'll describe it briefly to see if it rings any bells:
Imagine a dtml method called methodA that looks like this:
<!--#var standard_html_header--> Depth: <dtml-var "methodA(counter=counter+1)"> <!--#var standard_html_footer-->
Try this: <!--#var standard_html_header--> Depth: <dtml-var "methodA(_.None,_,counter=counter+1)"> <!--#var standard_html_footer-->
a) zope locks up/crashes because of the recursion b) methodA bails out because someone detects recursion or exceeds a stack limit of some sort
You will now see one of the two. Actually, DocumentTemplate will eventually raise a DTML call stack overflow exception, but you may think it's crashed because it might take a while to get the error. :)
What you *get*, however, is the very suprising:
Error type: KeyError Error value: standard_html_header
And if you rip out the reference to standard_html_..., you'll get the equally baffling:
Error type: NameError Error value: methodA
In practice, you'll find that this disappearing act occurs at about the third level of expression evaluation.
Ring any bells? It requires *horrible* workarounds when you run into it.
Horrible workarounds? Not really. Just make sure that if you're going to call any kind of DTML-ish thing with parameters, the first two positional parameters should be _.None and _ so that the current namespace is passed through. When you do a plain evaluation of such, the DTML machinery automatically passes them in, and you never see it happen. This is one of the really bad bits of DTML from a learning curve standpoint: it does magic behind the scenes for you that you really ought to be able to see so that you know how to do them when *you* need to do them.
"Phillip J. Eby" wrote:
Try this:
<!--#var standard_html_header--> Depth: <dtml-var "methodA(_.None,_,counter=counter+1)"> <!--#var standard_html_footer-->
Andrew Duncan suggested this one as well more or less as well - thanks. BUT! (read on...)
Horrible workarounds? Not really. Just make sure that if you're going to call any kind of DTML-ish thing with parameters, the first two positional parameters should be _.None and _ so that the current namespace is passed through.
Well, yes they were horrible; minus knowing that to descend more than two levels deep in evaluated method calls I must start passing these things, the workarounds *are* extremely ugly. So I'm glad I have this tidbit in my head now, but I've made a *lot* of compromises since :( And I continue to maintain that this is a *BUG* - here's why: (slightly modified version of the original example - emphasises bug issue) This version blows up: methodA: Depth: <dtml-var counter> <dtml-if "counter < 2"> <dtml-var "methodA(counter=counter+1)"> The error: Error type: KeyError Error value: counter This one works fine: methodA: Depth: <dtml-var counter> <dtml-if "counter < 1"> <dtml-var "methodA(counter=counter+1)"> Output is: Depth: 0 Depth: 1 I strongly believe this is a bug in that if it's the case that my namespace goes away minus passing in the namespace as a parameter, the second example *should not have worked*. It strikes me that someone went to the effort to ensure that namespace gets passed down for me to *some* extent, becuase I'm getting "Depth: 1". That it should fail at the next level appears entirely arbitrary. If this is really really not a bug, then we are going to have a piece of documentation that says: "If you want to call your methods within an expression context where you're calling depth is greater than two, you must pass in the namespace as (_.None,_[,...])" I would personally be embarassed to have such a note in there. Sorry, but this one has frustrated me way too many times. Please tell me why I'm wrong. Thanks, Dave Parker
Dave Parker wrote:
This version blows up: methodA: Depth: <dtml-var counter> <dtml-if "counter < 2"> <dtml-var "methodA(counter=counter+1)">
The error:
Error type: KeyError Error value: counter
Sorry - wrong screen - it is: Error type: NameError Error value: methodA One other thought (and I could be way off here), but Zope is responsible for taking action based on the fact that "" was specified - why can't the mechanism that takes that action just ensure for me that my namespace is propagated into whatever I might be calling, ad infinitum? It is certainly very counter-intuitive that it would not (very Perl like).
Dave Parker wrote:
"Phillip J. Eby" wrote:
Try this:
<!--#var standard_html_header--> Depth: <dtml-var "methodA(_.None,_,counter=counter+1)"> <!--#var standard_html_footer-->
Andrew Duncan suggested this one as well more or less as well - thanks. BUT! (read on...)
Horrible workarounds? Not really. Just make sure that if you're going to call any kind of DTML-ish thing with parameters, the first two positional parameters should be _.None and _ so that the current namespace is passed through.
Well, yes they were horrible; minus knowing that to descend more than two levels deep in evaluated method calls I must start passing these things, the workarounds *are* extremely ugly. So I'm glad I have this tidbit in my head now, but I've made a *lot* of compromises since :(
You are making this more mysterious than it really is. The rule doesn;t depend on the evaluation level. If you call a DTML method or document in an expression, then you must pass the namespace if you want it to be used. This arises from the fact that the expression syntax uses Python. This gives you more power at the cost of more work, a tradeoff.
And I continue to maintain that this is a *BUG* - here's why:
(slightly modified version of the original example - emphasises bug issue)
This version blows up: methodA: Depth: <dtml-var counter> <dtml-if "counter < 2"> <dtml-var "methodA(counter=counter+1)">
The error:
Error type: KeyError Error value: counter
This one works fine: methodA: Depth: <dtml-var counter> <dtml-if "counter < 1"> <dtml-var "methodA(counter=counter+1)">
Output is:
Depth: 0 Depth: 1
I strongly believe this is a bug in that if it's the case that my namespace goes away minus passing in the namespace as a parameter, the second example *should not have worked*.
Your analysis is flawed. Here's why. The method gets called recursively. The first time it is called, the namespace *is* passed in, by Zope. When it calls itself the second time, the namespace isn't there, because the method doesn't pass it. Lets work through this: In your second example, the method gets called. I assume that you arranged for counter to be initialized to zero. (I'm coming in at the tail end of this conversation. :) - On the first call, "Depth: 0" is output." The counter is < 1 so we enter the if block and call methodA. This succeeds because methodA is in the namespace. - On the second call, "Depth: 1" is output." Because counter is not <1, we never execute the 'if' and we're done. It doesn't matter that methodA is not in the namespace, because we never call it. Not consider your first example: - On the first call, "Depth: 0" is output." The counter is < 2 so we enter the if block and call methodA. This succeeds because methodA is in the namespace. - On the second call, "Depth: 1" is output." Because counter is < 2, we try to execute the if block. This fails (with a name error on methodA) because methodA is not in the namespace, because we never passed in the namespace.
It strikes me that someone went to the effort to ensure that namespace gets passed down for me to *some* extent,
That's right. When a method is called by Zope or when you insert it with the name attribute of the var tag: <dtml-var name=methodA> (aka <dtml-var methodA>) The namespace is passed in for you.
becuase I'm getting "Depth: 1". That it should fail at the next level appears entirely arbitrary.
It may appear that way, but it's not. :)
If this is really really not a bug, then we are going to have a piece of documentation that says:
"If you want to call your methods within an expression context where you're calling depth is greater than two, you must pass in the namespace as (_.None,_[,...])"
I would personally be embarassed to have such a note in there.
Right, that's why the documentation should say: "If you want to call your methods within an expression, you must pass in the namespace as (_.None,_[,...])"
Sorry, but this one has frustrated me way too many times. Please tell me why I'm wrong.
Hopefully, the explanation above helps. Jim -- Jim Fulton mailto:jim@digicool.com Technical Director (888) 344-4332 Python Powered! Digital Creations http://www.digicool.com http://www.python.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
Jim Fulton wrote:
Your analysis is flawed. Here's why. The method gets called recursively. The first time it is called, the namespace *is* passed in, by Zope. When it calls itself the second time, the namespace isn't there, because the method doesn't pass it. Lets work through this:
Blush - yup, you're right. So ok I guess I just have an "issue" in my head then, which is...
It strikes me that someone went to the effort to ensure that namespace gets passed down for me to *some* extent,
That's right. When a method is called by Zope or when you insert it with the name attribute of the var tag:
<dtml-var name=methodA> (aka <dtml-var methodA>)
The namespace is passed in for you.
...so why not take care of that for me *always*? You do it for me once, why not just keep on doing it? Put another way, what's the benefit to the user of parameterizing the namespace? - It's not at all intuitive. - In that I get namespace calling the function one way, but not another, it's inconsistent and therefore confusing - It adds a repetitiveness to the protocol that feels kludgy - From an end-user perspective, I feel like I'm getting around a *problem* with Zope's logic So it's a "user friendliness" issue. But I can guarantee you I'm not alone - we have a team of four doing heavy development in Zope and the *resounding* complaint is that Zope is rife with these apparent inconsistencies. I'm close to being done beating this issue, but while "apparent inconsistency" is in my head I'll thow out one I struggling with at this moment: If I do: <dtml-var "myMethod(value={'this':'that'})"> myMethod does not see "value" in the namespace whereas if I do: <dtml-var "myMethod(value=['this','that'])"> myMethod sees value just fine. So I can't pass in mapping objects?? Sigh. This sort of thing is *extremely* frustrating to someone just trying to crank out an app... Thanks for you time, and I apologize for all this ranting - I've been doing 18 hour days (at night). Dave
Dave Parker wrote:
Jim Fulton wrote:
Your analysis is flawed. Here's why. The method gets called recursively. The first time it is called, the namespace *is* passed in, by Zope. When it calls itself the second time, the namespace isn't there, because the method doesn't pass it. Lets work through this:
Blush - yup, you're right. So ok I guess I just have an "issue" in my head then, which is...
It strikes me that someone went to the effort to ensure that namespace gets passed down for me to *some* extent,
That's right. When a method is called by Zope or when you insert it with the name attribute of the var tag:
<dtml-var name=methodA> (aka <dtml-var methodA>)
The namespace is passed in for you.
...so why not take care of that for me *always*? You do it for me once, why not just keep on doing it?
The difference is level and environment. Expressions are low-level. They provide low-level access and control. With control comes resposnsibility.
Put another way, what's the benefit to the user of parameterizing the namespace?
It is consistent with Python. Since expressions are defined to be Python, the semantics are expected to carry through. In general, in Python, what you pass is what you get. OTOH, there is some precedence in Python for having things passed for you. In particular, the instance that a method is accessed through is passed automagically.
- It's not at all intuitive.
- In that I get namespace calling the function one way, but not another, it's inconsistent and therefore confusing
It's inconsistent because the languages are different. For example, '-' is legal in HTML, and therefore DTML, names but not in Pthon names.
- It adds a repetitiveness to the protocol that feels kludgy
Granted.
- From an end-user perspective, I feel like I'm getting around a *problem* with Zope's logic
In this case, the problem was with your understanding of Zope's logic. This sort of thing will be an issue for most technologies. OTOH, the fact that you didn't understand what was going on was, at least partly, Zope's fault. Although I think that the current situation is, on some level, justifiable, I'd also love to improve it. While I understand why it is necessary to pass "_.None, _", I'd love for it to be unnecessary. After reading your rant, I've given some thought to this and I think I might have an idea how to accomplish it. It does involve some, uh, hijinks under the hood, but I do think it is doable and am in favor of it myself. I'll put a request into the collector and will champion it with the Zope project management folks.
So it's a "user friendliness" issue. But I can guarantee you I'm not alone - we have a team of four doing heavy development in Zope and the *resounding* complaint is that Zope is rife with these apparent inconsistencies.
I'm close to being done beating this issue, but while "apparent inconsistency" is in my head I'll thow out one I struggling with at this moment:
If I do:
<dtml-var "myMethod(value={'this':'that'})">
myMethod does not see "value" in the namespace
Really? I can't reproduce this.
whereas if I do:
<dtml-var "myMethod(value=['this','that'])">
myMethod sees value just fine.
So I can't pass in mapping objects??
Yes you can. Something else is going on. I just tried an example like yours and it worked fine for me. I had method m1: <dtml-var value> and called it from m2: <dtml-var standard_html_header> <dtml-var "m1(value={'this':'that'})"> <dtml-var standard_html_footer> and got: {'this': 'that'} along with other decorations from the standard header and footer. Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
[Jim Fulton, on Fri, 17 Dec 1999] :: Although I think that the current situation is, on some level, :: justifiable, I'd also love to improve it. While I understand :: why it is necessary to pass "_.None, _", I'd love for it to :: be unnecessary. :: :: After reading your rant, I've given some thought to this and :: I think I might have an idea how to accomplish it. It does :: involve some, uh, hijinks under the hood, but I do think it is doable :: and am in favor of it myself. I'll put a request into the collector :: and will champion it with the Zope project management folks. You're scaring me. ;) Don't know what you have in mind, but if it involves diverging from predictable Pythonicness, I wonder if a better solution wouldn't be to just publish a simple tutorial on Python scoping rules and namespaces as part of the docs. The subject isn't that hard to grasp, it can be (as has been) covered in a one page treatment, but if it's laid out anywhere on the Zope Web site, I haven't seen it.
On Fri, 17 Dec 1999, Jim Fulton wrote:
Although I think that the current situation is, on some level, justifiable, I'd also love to improve it. While I understand why it is necessary to pass "_.None, _", I'd love for it to be unnecessary.
After reading your rant, I've given some thought to this and I think I might have an idea how to accomplish it. It does involve some, uh, hijinks under the hood, but I do think it is doable and am in favor of it myself. I'll put a request into the collector and will champion it with the Zope project management folks.
Nifty. Another issue that this may solve is the inconsistant way of calling methods. Originally, when writing GUF, the concept was the Zope administrator could use any Zope method to populate userlists, roles etc. However, as DTML methods require these magic variables, I had the choice of: - Using DTML calling conventions, and methods that could not fake it (Python Methods, External methods) would have to be called from a DTML stub. ZSQL methods are be the main one affected. - Put in a special case if meta_type == 'DTML Method', making the code and documentation more complex. -- ___ // Zen (alias Stuart Bishop) Work: zen@cs.rmit.edu.au // E N Senior Systems Alchemist Play: zen@shangri-la.dropbear.id.au //__ Computer Science, RMIT WWW: http://www.cs.rmit.edu.au/~zen
Dave Parker wrote:
Jim Fulton wrote:
Your analysis is flawed. Here's why. The method gets called recursively. The first time it is called, the namespace *is* passed in, by Zope. When it calls itself the second time, the namespace isn't there, because the method doesn't pass it. Lets work through this:
Blush - yup, you're right. So ok I guess I just have an "issue" in my head then, which is...
It strikes me that someone went to the effort to ensure that namespace gets passed down for me to *some* extent,
That's right. When a method is called by Zope or when you insert it with the name attribute of the var tag:
<dtml-var name=methodA> (aka <dtml-var methodA>)
The namespace is passed in for you.
...so why not take care of that for me *always*? You do it for me once, why not just keep on doing it?
The difference is level and environment. Expressions are low-level. They provide low-level access and control. With control comes resposnsibility.
Put another way, what's the benefit to the user of parameterizing the namespace?
It is consistent with Python. Since expressions are defined to be Python, the semantics are expected to carry through. In general, in Python, what you pass is what you get. OTOH, there is some precedence in Python for having things passed for you. In particular, the instance that a method is accessed through is passed automagically.
- It's not at all intuitive.
- In that I get namespace calling the function one way, but not another, it's inconsistent and therefore confusing
It's inconsistent because the languages are different. For example, '-' is legal in HTML, and therefore DTML, names but not in Pthon names.
- It adds a repetitiveness to the protocol that feels kludgy
Granted.
- From an end-user perspective, I feel like I'm getting around a *problem* with Zope's logic
In this case, the problem was with your understanding of Zope's logic. This sort of thing will be an issue for most technologies. OTOH, the fact that you didn't understand what was going on was, at least partly, Zope's fault. Although I think that the current situation is, on some level, justifiable, I'd also love to improve it. While I understand why it is necessary to pass "_.None, _", I'd love for it to be unnecessary. After reading your rant, I've given some thought to this and I think I might have an idea how to accomplish it. It does involve some, uh, hikinks, but should be doable. I'll submit this as a feature request to the collector and champion it with the product management folks here.
So it's a "user friendliness" issue. But I can guarantee you I'm not alone - we have a team of four doing heavy development in Zope and the *resounding* complaint is that Zope is rife with these apparent inconsistencies.
I'm close to being done beating this issue, but while "apparent inconsistency" is in my head I'll thow out one I struggling with at this moment:
If I do:
<dtml-var "myMethod(value={'this':'that'})">
myMethod does not see "value" in the namespace
Really? I can't reproduce this.
whereas if I do:
<dtml-var "myMethod(value=['this','that'])">
myMethod sees value just fine.
So I can't pass in mapping objects??
Yes you can. Something else is going on. I just tried an example like yours and it worked fine for me. I had method m1: <dtml-var value> and called it from m2: <dtml-var standard_html_header> <dtml-var "m1(value={'this':'that'})"> <dtml-var standard_html_footer> and got: {'this': 'that'} along with other decorations from the standard header and footer. Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
Jim Fulton wrote:
- From an end-user perspective, I feel like I'm getting around a *problem* with Zope's logic
In this case, the problem was with your understanding of Zope's logic. This sort of thing will be an issue for most technologies. OTOH, the fact that you didn't understand what was going on was, at least partly, Zope's fault.
Although I think that the current situation is, on some level, justifiable, I'd also love to improve it. While I understand why it is necessary to pass "_.None, _", I'd love for it to be unnecessary.
After reading your rant, I've given some thought to this and I think I might have an idea how to accomplish it. It does involve some, uh, hikinks, but should be doable. I'll submit this as a feature request to the collector and champion it with the product management folks here.
(REQUEST seems to need to be in there as well to preserve acquisition as well?) That's great news! Bearing in mind that I've been ignorant of REQUEST,_,_.None 'til yesterday, I'd guess that this change will answer about 80% of my issues with Zope. I don't know if I've mentioned it, but I really think Zope is among the most brilliantly concieved notions to come down the pike in a long long while. (I'm a reluctant ranter ;)
So I can't pass in mapping objects??
Yes you can. Something else is going on. I just tried an example like yours and it worked fine for me.
I had method m1:
<dtml-var value>
and called it from m2:
<dtml-var standard_html_header>
<dtml-var "m1(value={'this':'that'})">
<dtml-var standard_html_footer>
and got:
{'this': 'that'}
I think it had more to do with what I was trying to do with the value. I got a "Key Error", the meaning of which I probably misunderstood. Without attempting to pass the value in and attempting to do this: <dtml-let values="{'this':'that'}"> <dtml-in values> <dtml-var sequence-key> </dtml-in> </dtml-let> ...I get this: Error type: KeyError Error value: 0 Traceback (innermost last): File /home/caps/test/zope/lib/python/ZPublisher/Publish.py, line 165, in publish File /home/caps/test/zope/lib/python/ZPublisher/mapply.py, line 160, in mapply (Object: second) File /home/caps/test/zope/lib/python/ZPublisher/Publish.py, line 102, in call_object (Object: second) File /home/caps/test/zope/lib/python/OFS/DTMLMethod.py, line 145, in __call__ (Object: second) File /home/caps/test/zope/lib/python/DocumentTemplate/DT_String.py, line 502, in __call__ (Object: second) File /home/caps/test/zope/lib/python/DocumentTemplate/DT_Let.py, line 145, in render (Object: values="{'this':'that'}") File /home/caps/test/zope/lib/python/DocumentTemplate/DT_In.py, line 672, in renderwob (Object: values) KeyError: 0 So I was perhaps under the mistaken impression that I could iterate through mapping structures...? I strongly suspect this isn't a topic for the dev list anymore ;)
participants (5)
-
Dave Parker -
Jim Fulton -
Patrick Phalen -
Phillip J. Eby -
Stuart 'Zen' Bishop