[Zope-dev] References to hypertext templating languages
Don Hopkins
xardox@mindspring.com
Wed, 15 May 2002 10:45:23 -0700
From: "Chris Withers" <chrisw@nipltd.com>
Subject: Re: [Zope-dev] 'not:' kludgey?!
> Don Hopkins wrote:
> >
> > To be more accurate:
>
> Nope ;-)
>
> > SOME templating languages were never designed to be procedural
languages.
>
> ...if a language is designed to be a procedural language, then it ain't a
templating
> language...
>
> cheers,
>
> Chris
Please show me some references to where you got your definition of
"templating language".
Has the term been officiall defined by W3C, or is the term defined in the
literature?
References and examples, please!
A few years ago I implemented my own XML templating language which was a lot
like Lisp, based on the lessons of HyperTIES and other systems.
Since then I've decided to throw it away and reimplement an even better XML
templating language with Python, which is why this discussion interests me.
I've included some samples of my own XML templating language, including code
and data, at the end of this message.
Templating is a purpose to which you put a language. It's not an exclusive
category that rules out any particular programming technique.
Lisp is an excellent templating language for generating Lisp code, also
known as "metaprogramming'.
The C preprocessor is a woefully lame templating language, because it's
string based, and lacks procedural programming constructs.
But C++ templates are certainly a much more powerful templating language
than C preprocessor macros.
C++ templates allow you to trick the compiler into performing all kinds of
calculations for you. They're Turing complete, but astronomically complex.
Unfortunately you have to deeply understand almost everything about C++
templates in to use them (and especially to debug them). Case in point: STL.
However, Lisp macros are way more powerful than C++ templates, and much
easier to use, without requiring you to understand their inner workings.
Example: the "loop" macro.
Here are some references and examples of some of my earlier work on
hypertext templating languages.
During 1988-1990, I was working for Ben Shneiderman at the University of
Maryland Human Computer Interaction Lab, on a system called "HyperTIES":
designing, implementing and using a hypermedia markup language called
"HyperTIES Markup Language", with a browser for the NeWS window system, and
an authoring tool based on UniPress Emacs.
We considered using SGML at the time but decided not to, because it was too
complex and not very user-friendly, so we developed our own markup language.
Ben Shneiderman is very focused on the "ease of use" thing (for authors as
well as users), but as a researcher he also wants to experiment with
advanced techniques like authoring and browsing conditional and
parameterized hypermedia (aka "templates").
So HyperTIES was designed to be simple enough for non-programmers to author
and use, yet also powerful enough for programmers to make
application-specific templates and frameworks of macros, which
non-programmers could easily use to author their own pages, without
understanding the entire language.
The HyperTIES Markup Language syntax was designed to be simple ("low
syntactic surface area"), while also supporting abstraction, conditionals,
looping, variables, macros and other procedural programming language
constructs.
In 1989, the Unix version of HyperTIES supported java-like "applets" which
casual authors could easily embed and parameterize, without understanding
the implementation language.
I implemented the HyperTIES browser in C and NeWS PostScript, and the
authoring tool in Emacs Mocklisp, using James Gosling's NeWS window system
and Emacs text editor (he hadn't invented Java yet).
HyperTIES authors could use macros and other constructs like loops and
conditionals, to dynamically populate pages with interactive formatted text,
graphics, and parameterized applets (like pie menus, etc).
The HyperTIES browser would download a formatted PostScript display list to
the window server, then instantiate and customize all of the interactive
embedded widgets on the page.
It supported components such as encapsulated PostScript, graphical buttons,
pop-up shaped image maps, text editors, pie menus, and other interactive
widgets implemented in PostScript for the NeWS window system.
Casual hypermedia authors could create applets and pass parameters to them
without programming or understanding the PostScript implementation language
running in the window server.
Authors could also call on pre-defined higher level libraries of HyperTIES
macros, written by programmers to support their particular tasks, all
without knowing how to program the HyperTIES Markup Language running in the
browser.
You don't need to know how to program in a language, in order to simply call
pre-defined functions and macros.
It's misguided to artificially limit the power of a templating language, in
order to make it "easy to use".
Defining templating languages as "not procedural lanugages" is simply wrong,
and totally arbitrary.
That definition excludes many widely used, well known templating languages,
and totally ignores mountains of published literature and research on the
subject.
(Douglas Adams calls this the "Somebody Else's Problem" syndrome. Templating
languages that support procedural programming are so big, brightly painted
and obvious that they must be "Somebody Else's Problem", thus rendering them
invisible.)
Whether or not we agree on the definition of "templating" languages, I do
know that it's possible to design XML based languages that are both easy to
use for non-programmers to author, and also powerful enough for programmers
to extend to handle all kinds of tasks.
The power of abstraction makes it possible for programmers to create macros
that are easy for non-programmers to use, which can be illustrated by
HyperTIES and the anonymous XML templating languages included at the end of
this message. (Sorry, I never gave the language a name.)
It's unfortunate that some so-called "templating" languages leave out macros
and other powerful programming language constructs, so they are much less
useful than they would have been otherwise, which is a shame.
-Don
====
http://www.cms.dmu.ac.uk/General/hci/hcibib/HTML/HYPER/Shne91a.html
CS-TR-2433
Ben Shneiderman. Catherine Plaisant. Rodrigo Botafogo. Don Hopkins. William
Weiland. May 1991.
Designing to facilitate browsing: A look back at the Hyperties.
Since browsing hypertext can present a formidable cognitive challenge, user
interface design plays a major role in determining acceptability. In the
Unix workstation version of Hyperties, a research-oriented prototype, we
focussed on design features that facilitate browsing. We first give a
general overview of Hyperties and its markup language. Customizable
documents can be generated by the conditional text feature that enables
dynamic and selective display of text and graphics. In addition we
present: - an innovative solution to link identification: pop-out graphical
buttons of arbitrary shape. - application of pie menus to permit low
cognitive load actions that reduce the distraction of common actions, such
as page turning or window selection. - multiple window selection strategies
that reduce clutter and housekeeping effort. We preferred piles-of-tiles, in
which standard-sized windows were arranged in a consistent pattern on the
display and actions could be done rapidly, allowing users to concentrate on
the contents. (Also cross-referenced as CAR-TR-494) Human Computer
Interaction Laboratory, Center for Automation Research, Dept. of Computer
Science, Univ. of Maryland, Institute for Systems Research,
====
http://www.cs.umd.edu/hcil/hyperties/
Hypertext Research: The Development of HyperTIES
Starting in 1982, HCIL developed an early hypertext system on the IBM PC
computers. Ben Shneiderman invented the idea of having the text itself be
the link marker, a concept that came to be called embedded menus or
illuminated links. Earlier systems used typed-in codes, numbered menus or
link icons. Embedded menus were first implemented by Dan Ostroff in 1983 and
then applied and tested by Larry Koved (Koved and Shneiderman, 1986). In
1984-85 the work was supported by a contract from the US Department of
Interior in connection with the U.S. Holocaust Memorial Museum and Education
Center. Originally called The Interactive Encyclopedia Systems (TIES), we
ran into trademark conflicts and in 1986 changed the name to HyperTIES as we
moved toward commercial licensing with Cognetics Corporation. We conducted
approximately 20 empirical studies of many design variables which were
reported at the Hypertext 1987 conference and in array of journals and
books. Issues such as the use of light blue highlighting as the default
color for links, the inclusion of a history stack, easy access to a BACK
button, article length, and global string search were all studied
empirically. We used Hyperties in the widely circulated ACM-published disk
Hypertext on Hypertext which contained the full text of the 8 papers in the
July 1988 Communications of the ACM.
A Unix-based Sun implementation in the SUN OS and NEWS environments was
built by Don Hopkins, Bill Weiland, and Rodrigo Botafogo under the
leadership of Catherine Plaisant. Rodrigo Botafogo developed the first image
map implementation on the PC for touchscreen selections in the Smithsonian
Institution's exhibit "King Herod's Dream" in May 1988. The public access
kiosk, developed by Catherine Plaisant and Richard Potter, was called "Guide
to Opportunities in Volunteer Archaeology (GOVA) and toured for 18 months in
six cities.
In 1989, Ben Shneiderman and Greg Kearsley published Hypertext Hands-on!,
the world's first commercial electronic book with the highlighted links
idea. This innovative book/software package provided the first hands-on,
non-technical introduction to hypertext. The publicity claimed:
Highlights:
- describes hypertext applications for travel guides, product catalogs,
technical documentation, novels, blueprints, textbooks, encyclopedias, and
more
- software contains examples of a hyper novella, hyper travel guide, hyper
business procedures, hyper blueprints, and even a hyper joke.
- discusses system design issues such as user interface, performance,
networks, direct manipulation, windows, browsing, indexes, etc.
[...]
Other Hyperties features anticipated the World Wide Web. Charles Kreitzberg,
Whitney Quesenbery, and programmers at Cognetics made professional
implementations of image maps, animations, and a markup language called HTML
(Hyperties Markup Language). It is quite similar to the HTML markup language
used with web browsers; both drew on concepts in SGML, which continues to be
an important markup language within the publishing community. Hyperties also
had a Java-like scripting language, which allowed processes to be attached
to pages or to links.
Early readings on HyperTIES:
Ewing, J., Mehrabanzad, S., Sheck, S. Ostroff, D., and Shneiderman, B., An
experimental comparison of a mouse and arrow-jump keys for an interactive
encyclopedia, International Journal of Man-Machine Studies 24, 1 (January
1986), 29-45.
Shneiderman, B., User interface design for the Hyperties electronic
encyclopedia, Hypertext '87 Workshop Proceedings, Raleigh, NC, November
13-15, 1987, 199-204.
Shneiderman, B. (Editor), Hypertext on Hypertext, Hyperties disk with 1Mbyte
data and graphics incorporating Communication of the ACM (July 1988), New
York: ACM Press.
Koved, L. and Shneiderman, B., Embedded menus: Selecting items in context,
Communications of the ACM 29, 4 (April 1986), 312-318.
Shneiderman, B., Reflections on authoring, editing, and managing hypertext.
In Barrett, E. (Editor), The Society of Text, MIT Press, Cambridge, MA
(1989), 115-131.
Shneiderman, B. and Kearsley, G., Hypertext Hands-On! An Introduction to a
New Way of Organizing and Accessing Information, Addison-Wesley, Reading, MA
(1989).
Shneiderman, B., Plaisant, C., Botafogo, R., Hopkins, D., and Weiland, W.,
Designing to facilitate browsing: A look back at the Hyperties workstation
browser, Hypermedia 3, 2 (1991), 101-117.
Brethauer, D., Plaisant, C., Potter, R., and Shneiderman, B., Three
evaluations of museum installations of a hypertext system, Journal of the
American Society for Information Science 40, Special Issue on Hypertext,
(May 1989), 172-182.
====
This is a description I wrote to Dave Winer about my reaction to the
"open-sourcing" of Xanadu, which he asked me to post on his "Userland"
discussion board.
It describes the "info" hypertext system in UniPress Emacs, as well as
HyperTIES, HyperLook, and some ideas from other systems.
(I had to break it up into several messages to fit on the discussion board.)
http://static.userland.com/userLandDiscussArchive/msg010163.html
http://static.userland.com/userLandDiscussArchive/msg010164.html
http://static.userland.com/userLandDiscussArchive/msg010165.html
http://static.userland.com/userLandDiscussArchive/msg010166.html
http://static.userland.com/userLandDiscussArchive/msg010167.html
====
What follows are some examples from my own XML based hypertext templating
language.
Here is the URL of the experimental server, that you can play around with to
see how it works.
All the graphics and products are just test data, you can't acutally order
anything, and the credit card verification stuff is disabled.
http://www.lushcreations.tv/cgi-bin/mugger.cgi
Here is a typical page template, that processes a credit card entry form,
and relies on a whole bunch of macros and tables of xml data.
You can create your own forms by writing in a simple XML data format,
without doing any programming at all.
The author writes simple XML data and page templates, that calls the
pre-defined macros. The macros interpret the parameters and data, and render
the html.
No programming is necessary for authors to make page templates and forms,
but the language's support for programming was essential in order to
implement the task specific macros, which the author need not understand in
order to use.
The XML code can call named macros, that handle high level tasks like
rendering, handling and validating forms.
Macros can take attribute arguments (unordered of course ;-), as well the
XML structures they enclose, as parameters.
Most of the strangely named elements are macros, specific to the task of
generating html and processing forms, defined in terms of other macros and a
few built-in primitives.
<page name="Enter Credit Card">
<webpage title="Enter Credit Card">
<scope
formResult="0">
<validateform
secure="s"
name="CreditCard"
result="formResult"/>
<if><att>scope.formResult</att>
<then>
[Credit card validation and
processing stuff omitted.]
</then>
</if>
</scope>
</webpage>
</page>
Here is the "validateform" macro, which you don't have to understand in
order to use. Just pass it the name of a form, defined in XML below.
It calls on a bunch of other macros, and uses a framework of form, widget
and validator definitions, also written in XML (below).
<macro name="validateform" secure="">
<with
collection="forms"
element="form"
key="name"
value="{validateform.name}">
<scope
valid="true"
errormessages=""
varval=""
textsize="64"
textrows="64"
textcols="8">
<CENTER>
<postform
secure="{validateform.secure}">
<TABLE>
<foreach
element="form">
<if><cgidefined name="{varname}"/>
<then>
<def element="scope" key="varval"><cgi name="{varname}" default=""
remove="false"/></def>
<def element="session" key="{varname}"><att>scope.varval</att></def>
</then>
<else>
<def element="scope" key="varval"><att>session.{varname}</att></def>
</else>
</if>
<if><isempty><att>validate</att></isempty>
<then>
<displaywidget/>
</then>
<else>
<if><strip><expandchildren
collection="validators"
element="validator"
key="name"
value="{validate}"/></strip>
<then>
<hidden
NAME="{varname}"
VALUE="{varval}"/>
<displaywidget/>
<def element="session" key="{varname}">{varval}</def>
</then>
<else>
<def element="scope" key="valid">false</def>
<def element="scope" key="errormessages">{scope.errormessages}
<H2>{error}</H2></def>
<TR><TD COLSPAN="2">
<H2>
{error}
</H2>
</TD></TR>
<requestwidget/>
</else>
</if>
</else>
</if>
</foreach>
<if><eq><att>scope.valid</att>
<quote>true</quote></eq>
<then>
<TR>
<TD COLSPAN="2">
<CENTER>
<H1>
Thank you!
</H1>
<store key="{validateform.result}">1</store>
</CENTER>
</TD>
</TR>
</then>
<else>
<TR>
<TD COLSPAN="2">
<CENTER>
<H1>
Please try again!
</H1>
<inlinenavbutton name="Enter Corrections" page="Enter Credit Card"/>
<store key="{validateform.result}">0</store>
</CENTER>
</TD>
</TR>
</else>
</if>
</TABLE>
</postform>
<navbutton name="Return to Form"
page="Credit Card" secure="s">
<hideform
name="{validateform.name}"/>
</navbutton>
</CENTER>
</scope>
</with>
</macro>
Here are the XML tables that the form processing framework uses to render,
submit and validate the forms.
I've omitted some parts like the names of all the states, etc, so you can
get the idea without reading all the code.
All of the information needed to completely handle a form is specified on
one place, in XML.
In order to make a form, you only need to understand the <form> and <widget>
syntax, and the framework handles the rest.
The framework (implemented with macros) takes care of all the heavy lifting
and book keeping required to process the forms, by manipulating data in the
form of XML structures, and calling on primitives defined in the language
interpreter.
This is called "data driven programming", which is a common technique with
lisp-like languages, that allow you to cleanly and easily embed code inside
of data, nest data inside of code, pass data and code around as parameters,
evaluate, deconstruct and reconstruct code as data, etc.
<forms>
<form name="NewAccount">
<widget type="text" label="Name" varname="newuser"
srcvar="user.name" validate="isnotempty" error="Please enter your name."/>
<widget type="password" label="Password"
varname="newpassword1" validate="isnotempty" error="Please enter the same
password twice."/>
<widget type="password" label="Repeat Password"
varname="newpassword2" validate="isnewpassword1" error="Please enter the
same password twice."/>
<widget type="text" label="Email Address" varname="email"
srcvar="user.email" validate="isany" error="Please enter your email
address."/>
<widget type="text" label="Postal Address" varname="address"
srcvar="user.address" validate="isany" error="Please enter your shipping
address."/>
<widget type="text" label="City" varname="city"
srcvar="user.city" validate="isany" error="Please enter your city."/>
<widget type="text" label="State" varname="state"
srcvar="user.state" validate="isany" error="Please enter your state."/>
<widget type="text" label="Zip Code" varname="zipcode"
srcvar="user.zipcode" validate="isany" error="Please enter your zip code."/>
<widget type="text" label="Country" varname="country"
srcvar="user.country" validate="isany" error="Please enter your country."/>
<widget type="text" label="Phone" varname="phone"
srcvar="user.phone" validate="isany" error="Please enter your phone number,
including the area code."/>
<widget type="text" label="Fax" varname="fax"
srcvar="user.fax" validate="isany" error="Please enter your fax phone
number, including the area code."/>
</form>
<form name="CreditCard">
<widget type="header" label="Billing and Shipping Form"/>
<widget type="subrule"/>
<widget type="subheader" label="Billing Address"/>
<widget type="menu" label="Credit Card" varname="cardtype"
default="Visa" validate="iscardtype" error="Please select a valid credit
card type.">
<item label="Visa" value="Visa"/>
<item label="MasterCard" value="MasterCard"/>
<item label="American Express"
value="AmericanExpress"/>
<item label="Choice" value="Choice"/>
<item label="Discover" value="Discover"/>
</widget>
<widget type="menu" label="Expiration Month"
varname="cardexpirationmonth" validate="ismonth" default="01" error="Please
select a valid credit card expiration month.">
<item label="January" value="01"/>
[...]
</widget>
<widget type="text" label="Card Number" varname="cardnumber"
validate="iscardnumber" secret="1" error="Please enter a valid credit card
number."/>
<widget type="text" label="Name on Card" varname="cardname"
srcvar="user.name" validate="isnotempty" error="Please enter the name on the
credit card."/>
<widget type="text" label="Billing Address"
varname="cardaddress" srcvar="user.address" validate="isnotempty"
error="Please enter the credit card billing address."/>
<widget type="text" label="City" varname="cardcity"
srcvar="user.city" validate="isnotempty" error="Please enter the city of the
credit card billing address."/>
<widget type="menu" label="State" varname="cardstate"
srcvar="user.state" validate="isstate" default="CA" error="Please select a
valid state.">
<item label="None" value=""/>
<item label="AK" value="AK"/>
[...]
</widget>
<widget type="text" label="Zip Code" varname="cardzipcode"
srcvar="user.zipcode" validate="isnotempty" error="Please enter the zip code
of the credit card billing address."/\
>
<widget type="text" label="Country" varname="cardcountry"
srcvar="user.country" validate="isnotempty" error="Please enter the country
of the credit card billing address."/>
<widget type="text" label="Phone" varname="cardphone"
srcvar="user.phone" validate="isphone" error="Please enter the billing phone
number, including the area code."/>
<widget type="text" label="Fax" varname="cardfax"
srcvar="user.fax" validate="isphoneorempty" dstvar="cardfax" error="Please
enter the billing fax phone number, including th\
e area code."/>
<widget type="space"/>
<widget type="subheader" label="Shipping Address"/>
<widget type="text" label="Name Of Recipient"
varname="shippingname" srcvar="user.name" validate="isnotempty"
error="Please enter the name of the recipient."/>
<widget type="text" label="Shipping Address"
varname="shippingaddress" srcvar="user.address" validate="isnotempty"
error="Please enter the recipient's shipping address."/>
<widget type="text" label="City" varname="shippingcity"
srcvar="user.city" validate="isnotempty" error="Please enter the city of the
recipient's shipping address."/>
<widget type="menu" label="State" varname="shippingstate"
srcvar="user.state" validate="isstate" default="CA" error="Please select a
valid state.">
<item label="None" value=""/>
<item label="AK" value="AK"/>
[...]
</widget>
<widget type="text" label="Zip Code"
varname="shippingzipcode" srcvar="user.zipcode" validate="isnotempty"
error="Please enter the zip code of the recipient's shipping addre\
ss."/>
<widget type="text" label="Country"
varname="shippingcountry" srcvar="user.country" validate="isnotempty"
error="Please enter the country of the recipient's shipping address\
."/>
<widget type="text" label="Phone" varname="shippingphone"
srcvar="user.phone" validate="isphone" error="Please enter the recipient's
phone number, including the area code."/\
>
<widget type="text" label="Fax" varname="shippingfax"
srcvar="user.fax" validate="isphoneorempty" dstvar="cardfax" error="Please
enter the recipient's fax phone number, incl\
uding the area code."/>
<widget type="rule"/>
</form>
The widget types themselves are defined in another XML table. Each widget
consists of "hide", "display" and "request" handlers, that are defined in
XML of course.
<widgets>
<widget type="text">
<scope varval="">
<getvarval/>
<switch val="{formcommand}">
<case val="hide">
<hidden
NAME="{widget.varname}"
VALUE="{scope.varval}"/>
</case>
<case val="display">
<TR>
<formlabel>{widget.label}</formlabel>
<TD>
<showerrmsg/>
<if><att>widget.secret</att>
<then>
<censorsecret>{scope.varval}</censorsecret>
</then>
<else>
{scope.varval}
</else>
</if>
</TD>
</TR>
</case>
<case val="request">
<TR>
<formlabel>{widget.label}</formlabel>
<TD>
<showerrmsg/>
<INPUT
TYPE="TEXT"
SIZE="{textsize}"
NAME="{widget.varname}"
VALUE="{scope.varval}"/>
</TD>
</TR>
</case>
</switch>
</scope>
</widget>
[...]
<widget type="menu">
<scope varval="">
<getvarval/>
<switch val="{formcommand}">
<case val="hide">
<hidden
NAME="{widget.varname}"
VALUE="{scope.varval}"/>
</case>
<case val="display">
<TR>
<formlabel>{widget.label}</formlabel>
<TD>
<showerrmsg/>
<foreach
element="widget">
<ifelement
name="item">
<if><eq><att>item.value</att>
<att>scope.varval</att></eq>
<then>{item.label}</then>
</if>
</ifelement>
</foreach>
</TD>
</TR>
</case>
<case val="request">
<TR>
<formlabel>{widget.label}</formlabel>
<TD>
<showerrmsg/>
<SELECT
NAME="{widget.varname}">
<foreach
element="widget">
<ifelement name="item">
<OPTION VALUE="{item.value}">{item.label}
<selectif>
<eq><att>item.value</att>
<att>scope.varval</att></eq>
</selectif>
</OPTION>
</ifelement>
</foreach>
</SELECT>
</TD>
</TR>
</case>
</switch>
</scope>
</widget>
<widget type="header">
<switch val="{formcommand}">
<case val="hide">
</case>
<case val="display">
<TR>
<TD> </TD>
<TD>
<CENTER>
<H1>{widget.label}</H1>
</CENTER>
</TD>
</TR>
</case>
<case val="request">
<TR>
<TD> </TD>
<TD>
<CENTER>
<H1>{widget.label}</H1>
</CENTER>
</TD>
</TR>
</case>
</switch>
</widget>
[...]
</widgets>
And here are some of the validators, naturally implemented in XML, but
calling on a few primitives implemented in C.
<validators>
<validator name="isany">
1
</validator>
<validator name="isnotempty">
<isnotempty><strip><att>session.{varname}</att></strip></isnotempty>
</validator>
<validator name="isnewpassword1">
<eq><att>session.{varname}</att>
<att>session.newpassword1</att></eq>
</validator>
<validator name="isyear">
<and><nge><att>session.{varname}</att>
<quote>0</quote></nge>
<nle><att>session.{varname}</att>
<quote>99</quote></nle></and>
</validator>
<validator name="ismonth">
<and><nge><att>session.{varname}</att>
<quote>1</quote></nge>
<nle><att>session.{varname}</att>
<quote>12</quote></nle></and>
</validator>
[...]
</validators>