[ZDP] The writing bug

[Martijn Faassen]

Hi there,

I was reading Tom's piece on Zope when the writing bug caught me. I
wrote down why I think web application platforms are useful and
important. Now that I have the piece I don't quite know what to do with
it; I'm open to suggestions. :)

Enjoy (I hope),

Martijn

----

A Web Application Platform -- Why?

Zope is a web application platform. What is a web application platform
and what can you use one for? For that we need to look more closely at
the history of the world wide web. Originally, web servers mainly
served static HTML pages. A static HTML page is a text file that
resides somewhere on a web server. The contents of a static page does
not change dynamically; it's just an inert file on the server. With
static web pages, there is no sophisticated interaction possible with
the person that is viewing them; you click on hyperlinks to request
new pages, but that's just about it.

Since HTML pages are plain text, however, many people immediately
realized that it is easy to create software that can generate HTML
pages automatically. You can run a program on the server that for
instance gathers information from some database, and creates a new
static web page out of it regularly. This way pages could be updated
more quickly. Fairly complicated datasets could be published on the
web by automatically generating the pages for them.

Web page generation software can be written in any programming
language; the language that was used most often however was Perl. Perl
is a programming language that has very sophisticated text processing
abilities, and was already installed on many Unix-based web
servers. It was therefore natural that people would start to use
programs written in Perl (Perl scripts) to generate web pages.

The next step in the evolution of the web was to allow the people
browsing the web to also enter new data, on forms. A search engine is
a well-known example of this; a user can type in a phrase in an entry
box, press the 'submit' button, and the phrase is sent to the server
that runs the search engine. The server then generates a web pages on
the fly (for instance the search results). A protocol was designed to
allow the web server to send such submitted data to a program, and for
the program to respond; the Common Gateway Interface, or CGI.

Now a new sense of dynamism was brought to the web. With CGI, you can
send information to a web server, and the web server has to respond
instantly. Web pages that exist as static text, or even automatically
generated static text, are not enough for this; these pages need to be
created dynamically. They're still HTML text, but they don't exist as
files on the web server anymore -- there are just programs on the
server that create the HTML to send to you. Again, Perl was one of the
favorite language to use to write these programs.

The state of the web now is for a large part a mixture of static
pages, automatically generated but still static pages, and scripts
that dynamically generate pages. Most web servers have a mixture of
these types of pages. This works. Many web sites are run this way. It
is also a development and maintenance nightmare.

Static pages are difficult to create and maintain, mainly because of
the repetitiveness. Pages on web sites need to conform to some local
theme; there's a navigation bar on the top of all pages, for instance,
or perhaps an email address of the webmaster at the bottom. The color
scheme of all the pages should look the same. The same types of fonts
should be used on each page. It is very tedious to make sure all your
pages conform to the theme; it is therefore also prone to errors. You
might update one page when the navigation bar changes (perhaps it has
a new link), but forget to update another. If you want to turn all
your web pages purple to protest against the mistreatment of
dinosaurs, you need to edit the HTML code for each page to change
their background color.

Luckily many sophisticated authoring tools exist that can help you
with this process. Even with them, however it is hard to build a
consistent site, especially if more than one people works to build the
site. But it's managable -- until you need to start dealing with
dynamic pages.

A dynamic page is generated by a program. Not everybody knows how to
modify programs. Especially not if they're Perl scripts as they
usually are -- Perl code has a deserved reputation of being rather
hard to read, especially for people that haven't immersed themselves
into the language. People who create web content or layout often
haven't immersed themselves into Perl or some other language; it's not
their speciality. How do they make these programs conform? Even an
experienced programmer may have difficulty; programmers know it's hard
to maintain other people's code.

There is another problem with the scripts solution for a dynamic
web. Each script may do the same thing (respond to users, fetch data,
and generate a web page) in a slightly different way. Each different
way makes a new, different bug, inconsistency or security hole
possible. Although within limits it is nice that people can solve the
same problems in different ways (as each way has its own advantages
and disadvantages), if every script has to re-invent the wheel all
over, it is far more work to write them and debug them, and to make
them work together. It also becomes a problem later, when they need to
be maintained.

There is a trend in the development of the web. At first, web sites
mainly published static pages. This can be compared to what magazines
do. But we're dealing with the Internet here, not
magazines. Interactivity and communication are the main things that make
the net so powerful and appealing. After all, computers can do more
than glossy paper! Computers run *applications*, not magazines. So it
is natural that people expect that web sites to be more like
applications. A dynamic web sites behaves like an application; both on
the end user side (where users interact with the site), and on the
side of the people who provide the contents for the site -- and with
the Internet often there's an overlap between these two parties.

A web application platform is a piece of web server software that
enables you to create these flexible and dynamic web sites of the
future. A good web application platform should offer integrated and
uniform ways to respond to user requests, fetch data from a database,
and generate pages on the fly. It should make the development
maintainance of a dynamic and consistent web site less difficult, and
more consistent. There should be an integrated security system. It
should be possible for the layout designer and programmer to work
together without being experts in each other fields. And most
importantly, a web server application platform should be powerful and
flexible -- many types of web applications are possible, after all.

Zope is a very good web server application platform.