Using DTML is too difficult (was: [Zope] DTML Syntax contd. + rant)
<snip> Martijn, I _really_ like Zope, but I am afraid I have to agree with you. I'll add some more rant. I believe it's important to stress that this is _not_ meant as criticism per se, but only to point out what even for people who have used ZOpe for almost a year are major flaws in its usability. And I can vote for Martijn, that he's not stupid; I myself have the excuse of not coming from a computer science background, but still I rabble on.
Philosophical end rant:
If I ever get the time I'd be tempted to work on a DTML 'cleanup' project. DTML is currently *far* too much like Perl and not enough like Python. *some* easy things are very easy, but as a consequence some other easy things become far too hard, or at least look far too complicated. You can spell the same thing in too many ways. The community encourages and praises additions for convenience (like the 'default' option, or the new extended &entity; syntax) but as a result DTML loses its conceptual integrity. It becomes too big and not easy to understand.
I understand the argument that DTML shouldn't be used for complicated purposes, and that you should use Python. This is fine and good, but in practice people *do* use DTML for complicated purposes. ZClasses in fact encourage this.
The point is that it's not always clear when things are complicated. And using python for complicated tasks is not exactly a piece of cake either, for a number of reasons. - writing products in python is complicated and mostly not worth it - writing python base classes for ZClasses is easier, but using them in the Z framework requires quite a bit of understanding of ZClasses and the Zope framework in general - External methods are much easier, but they have their own disadvantages: (1) it is less than obvious how they interact with the dtml methods/namespace they are called from (2) they are not easily transported to another machine, because you need physical access to the filesystem for that (I understand that that's the only way)
DTML should be more like Python.
Yes
Luckily the Zope framework does allow new objects to be plugged in using something else than DTML, so not all hope is lost. :)
But it should be easier to use this stuff (do I hear the word developer documentation??? uuummm)
Feedback on this rant would be appreciated.
My 2 cents. Rik
Rik Hoekstra writes:
(quoting Martijn)
If I ever get the time I'd be tempted to work on a DTML 'cleanup' project. DTML is currently *far* too much like Perl and not enough like Python. *some* easy things are very easy, but as a consequence some other easy things become far too hard, or at least look far too complicated. You can spell the same thing in too many ways. The community encourages and praises additions for convenience (like the 'default' option, or the new extended &entity; syntax) but as a result DTML loses its conceptual integrity. It becomes too big and not easy to understand.
Amusingly, Greg Ward, the resident Perl expert here, made the same comparison after working with Zope for a while, and I can see his point. There are lots of default behaviours that usually do the Right Thing, but when they aren't what you want, it can be hard to figure out what's going on. When things do go wrong, you don't have a lot of diagnostic tools to pin down the problem; what's my namespace stack look like? Developing Zope stuff, I find I'm often inserting "return repr(someobject)" into ExternalMethods to check on variable values; a Web-based namespace + variable browser would be helpful. DTML is beautiful when the problem maps nicely to existing DTML constructs; however, once you start wanting to add lots of complicated behaviour, you have to start writing Python code, and for writing Python code, the Zope development environment has drawbacks, ably listed by Rik:
- writing products in python is complicated and mostly not worth it - writing python base classes for ZClasses is easier, but using them in the Z framework requires quite a bit of understanding of ZClasses and the Zope framework in general - External methods are much easier, but they have their own disadvantages: (1) it is less than obvious how they interact with the dtml methods/namespace they are called from (2) they are not easily transported to another machine, because you need physical access to the filesystem for that (I understand that that's the only way)
Security is, I think, part of the reason that DTML is a sandbox; if someone gets your Zope management password, they can edit your pages and run SQL commands in your database, but they can't do 'rm -rf /', because they can't install ExternalMethods or classes without access to the machine's filesystem, and there's no way to do such things from DTML. I'm not sure what the fix is: a hypothetical ngDTML, which is different from DTML and would provide more of Python's power? Better debugging tools and support with Zope? Better support for developing and debugging your code outside of the Zope server? I think the solution lies more with the latter two options; ngDTML would be difficult to design, and it might be easy to repeat the same mistakes. Let me hasten to add that this isn't Zope-bashing. We also use Java servlets using Apache's mod_jserv. Servlets have all of the same problems, with some additional ones: 1) Instead of a fast interpreter, you get to run the slow "javac" every time you make a change. 2) No standard solution for HTML templating. Sun has JSP, but they mix your Java code with the HTML, which is a design mistake that we're trying to get away from! 3) Instead of writing L = ['material', 'substrate'] in Python, you get to write: import java.lang.Vector; ... Vector L = new Vector(); L.addElement( "material" ); L.addElement( "substrate" ); (Why people think a systems language like Java is appropriate for Web stuff is beyond me.) 4) All those useful #in and #tree tags? Not there -- start writing your own! 5) SQL queries? Start writing JDBC code... try { ResultSet rs = doSelect(query); if (rs.next()) return new Integer(rs.getInt(1)); } catch (SQLException e) { throw new DatabaseCriticalException("Error accessing ResultSet in SequenceWrapper.currentVersion().\nQuery: " + qry + "\nException: " + e.getMessage()); } <!--#in myQuery--> is much easier, obviously. Let me be clear; we're finding that Zope blows other tools out of the water. (If I were to get a management edict that "Zope is out; tool X is in", I'd start tidying up my CV. Such an edict is unlikely, thankfully.) There's just still lots of room for improvement, and we all really care about that. :) -- A.M. Kuchling http://starship.python.net/crew/amk/ "I must confess, I have always wondered what lay beyond life, my dear." "Yeah, everybody wonders. And sooner or later everybody gets to find out." -- Norton I and Death, in SANDMAN #31: "Three Septembers and a January"
Ah, more people replying to my rant. Thanks! "Andrew M. Kuchling" wrote: [I say DTML is too much like Perl]
Amusingly, Greg Ward, the resident Perl expert here, made the same comparison after working with Zope for a while, and I can see his point. There are lots of default behaviours that usually do the Right Thing, but when they aren't what you want, it can be hard to figure out what's going on.
Right. And often things are spelled in rather nasty ways. Note that it is great that you *can* spell them at all. DTML is undoubtely extremely powerful.
When things do go wrong, you don't have a lot of diagnostic tools to pin down the problem; what's my namespace stack look like? Developing Zope stuff, I find I'm often inserting "return repr(someobject)" into ExternalMethods to check on variable values; a Web-based namespace + variable browser would be helpful.
As another trick you can raise an exception. I've used this to debug products and such. A nice browser would definitely help, though.
DTML is beautiful when the problem maps nicely to existing DTML constructs;
Agreed. Today my coworker wanted a list sorted alphabetically. That was just a question of adding 'sort=title' to the in tag and it worked. About a minute work. That's great. We shouldn't lose that. [discussion of drawbacks of writing Python] I comment on this in my reply to Rik's post.
Security is, I think, part of the reason that DTML is a sandbox; if someone gets your Zope management password, they can edit your pages and run SQL commands in your database, but they can't do 'rm -rf /', because they can't install ExternalMethods or classes without access to the machine's filesystem, and there's no way to do such things from DTML.
Don't forget Zope's powerful security delegation features as well. Someone who is only a local manager is unable to mess up the rest of the site. DTML security is good here too.
I'm not sure what the fix is: a hypothetical ngDTML, which is different from DTML and would provide more of Python's power?
I think a start from scratch with a 'ngDTML' would definitely be a good approach, in time. It could still look and feel very similar to current DTML, but be more consistent and simpler. Current powerful but adhoc constructs should be generalized or moved into a 'library' somehow.
Better debugging tools and support with Zope?
This would help too.
Better support for developing and debugging your code outside of the Zope server?
This would help too.
I think the solution lies more with the latter two options;
Definitely the latter two options, combined with more and better documentation would be the best short term approach, but I don't think the current evolution of DTML is good in the long term.
ngDTML would be difficult to design, and it might be easy to repeat the same mistakes.
Definitely it wouldn't be an easy thing to design. But I think this is a solution we should definitely look at for the longer term.
Let me hasten to add that this isn't Zope-bashing. [snip]
This isn't either. Zope is cool! Regards, Martijn
Hi, Noting this interesting discussion I thought I might butt in with some ideas. Firstly, Python Methods are *really* cool! By making them with parameters of [self, REQUEST] you can use them as drop-in replacements for DTML methods/Docs. A lot of the disadvantages mentioned are solved in Python Methods. With regards to a DTML replacement you may wish to look at the ideas expressed in the MFI, XMLWidgets and my never-ending project - VisualZope. VZ works by having a XML Form with namespaces (almost) implemented. Then, by defining widgets in either Zclass/Repository or in the acquisition path of the Form you can define how Form tags are interpreted. This is so useful that I've added a simple "intelligent" version of dtml-var to allow XML forms to be merged together. Basically, Forms are like DTML Documents except that they provide a structured way of modifing other parts of the Form than the area indicated by the tag. So, you can "include" a widget and the divisions will be created, the css sheets merged in and the javascript client-side DOM will be added all automatically. They will all be placed in the correct place without having multiple <dtml-vars>. We *NEED* behaviours (the equivalent of the MFI selection system I think). For example, if you add the "window" behaviour to the tag above then the behaviour will add in css,division additions to make it look like a MDI window and add javascript code to the constructor to make it draggable, resizable and hiddable. If you then add "ZAMLEditor" behaviour then you can use the XMLWidgets style of editing to edit the contents of this window. Yet another idea is to add the "theme-beos" behaviour before the window behaviour to make the window use the pixmaps etc in the beos theme. By the way - can DigiCool add ZWiki to the Zope.org site? This would be *really* cool for adding group scratchpads for various projects. Cheers, Anthony Pfrunder Computer Systems Engineering University of Queensland
Anthony Pfrunder wrote:
Firstly, Python Methods are *really* cool! By making them with parameters of [self, REQUEST] you can use them as drop-in replacements for DTML methods/Docs. A lot of the disadvantages mentioned are solved in Python Methods.
Not to mention the ability to do some really cool stuff with ZCatalogs (find all items beginning with the letter 'A', searching using metaphone or soundex [I did this, if anyon'es interested], etc.) I can't figure how to do the <dtml-with> tag in python, though. -- Itamar - itamars@ibm.net
Itamar Shtull-Trauring wrote:
Not to mention the ability to do some really cool stuff with ZCatalogs (find all items beginning with the letter 'A', searching using metaphone or soundex [I did this, if anyon'es interested], etc.) I can't figure how to do the <dtml-with> tag in python, though.
For some uses (such as <dtml-with "_(x=3, ...)">), you can substitute local variables. For example, something like <dtml-with "ick.arg.blah"> <dtml-var thingy> <dtml-call amethod> </dtml-with> could become it = ick.arg.blah print it.thingy it.amethod() Pushing namespaces onto the stack for the use of called methods is quite another matter. You would need code like try: _._push(ick.arg.blah) ... finally: _.pop() and of course this won't work since _push and _pop are private methods. Even if they were properly exposed, I'm not sure how this could be made pretty. Cheers, Evan @ 4-am
On Thu, 11 Nov 1999, Itamar Shtull-Trauring wrote: [snip]
Not to mention the ability to do some really cool stuff with ZCatalogs (find all items beginning with the letter 'A', searching using metaphone or soundex [I did this, if anyon'es interested], etc.) I can't figure how to do the <dtml-with> tag in python, though.
<dtml-with MyStuff> <dtml-var coolstuff> </dtml-with> --> self.MyStuff.coolstuff(REQUEST) The other way is to "emulate" a dictionary (which is all REQUEST is) and pass that to your Python Method and then use it to look things up. The (yet) other way is to "emulate" an __of__ relationship in the base class and then every attribute acquisition is passed to your class and you can do what you wish in the ZClasses Python Method. Cheers, Anthony Pfrunder Computer Systems Engineering University of Queensland
Hi people, Ah, you're replying to my rant here. :) Rik Hoekstra wrote: [snip]
I _really_ like Zope,
me too, no question.
but I am afraid I have to agree with you.
I agree with myself too. :)
I'll add some more rant. I believe it's important to stress that this is _not_ meant as criticism per se, but only to point out what even for people who have used ZOpe for almost a year are major flaws in its usability. And I can vote for Martijn, that he's not stupid; I myself have the excuse of not coming from a computer science background, but still I rabble on.
Thanks for your vote of non-stupidity. :)
Philosophical end rant:
If I ever get the time I'd be tempted to work on a DTML 'cleanup' project. DTML is currently *far* too much like Perl and not enough like Python. *some* easy things are very easy, but as a consequence some other easy things become far too hard, or at least look far too complicated. You can spell the same thing in too many ways. The community encourages and praises additions for convenience (like the 'default' option, or the new extended &entity; syntax) but as a result DTML loses its conceptual integrity. It becomes too big and not easy to understand.
I understand the argument that DTML shouldn't be used for complicated purposes, and that you should use Python. This is fine and good, but in practice people *do* use DTML for complicated purposes. ZClasses in fact encourage this.
The point is that it's not always clear when things are complicated. And using python for complicated tasks is not exactly a piece of cake either, for a number of reasons. - writing products in python is complicated and mostly not worth it
Though this is getting easier in the newer Zopes. Or I'm learning more about it. I highly recommend the 'Boring HOWTO' in this regard, by the way.
- writing python base classes for ZClasses is easier, but using them in the Z framework requires quite a bit of understanding of ZClasses and the Zope framework in general
Yes, this is pretty easy, but (as Amos remarked in his slides), distribution is currently difficult. Also if something goes wrong you can end up with objects in weird states which you can't delete. And it's true they could probably be simplified (or the documentation improved) so that still less understanding about Zope's inner details is necessary.
- External methods are much easier, but they have their own disadvantages: (1) it is less than obvious how they interact with the dtml methods/namespace they are called from (2) they are not easily transported to another machine, because you need physical access to the filesystem for that (I understand that that's the only way)
Though you can say the same for Products and ZClasses for Python base classes. Python Methods are cool. I haven't played a lot with them as I'm working on a production site, but I intend to play with them more.
DTML should be more like Python.
Yes
Luckily the Zope framework does allow new objects to be plugged in using something else than DTML, so not all hope is lost. :)
But it should be easier to use this stuff (do I hear the word developer documentation??? uuummm)
It's not only documentation though. The 'with' issue that caused me to rant is simply not consistent enough. There are more places in DTML where it is like this. The _[_['sequence-item']] style constructions are also obfuscatory.
Feedback on this rant would be appreciated.
My 2 cents.
Thanks! Regards, Martijn
participants (6)
-
Andrew M. Kuchling -
Anthony Pfrunder -
Evan Simpson -
Itamar Shtull-Trauring -
Martijn Faassen -
Rik Hoekstra