Howdy folks. As advertised in my last email, below is our Fishbowl Proposal on a new template architecture. Some notes: 1) This mostly replaces DTML. As mentioned, the goal isn't necessarily to get DTML removed from Zope. Rather, most of the audience will see it little or none of the time. 2) This proposal has gotten more internal DC scrutiny from a requirements perspective than anything we've done. The proposal has been rewritten four times. 3) The original version of this forced the connection between reinventing the data tier (documents) and the presentation tier (templates/pages/stylesheets). We eventually found the way to decouple the connection, but we still need to write a documents proposal. 4) What this really, really needs right now is extensive user documentation with screenshots written *before* the software. This is where we got stuck. Docs are the fastest way to confront the myriad of usage details. 5) Our proposal is a little more ambitious than HiperDOM (we have a concept of "components" that are hinted to but not explained). But for now, HiperDOM is a good basis for exploring XHTML Templates. 6) I'm still not convinced template is the right word. Unfortunately I'm less convinced at any alternatives. Whatever is chosen, it must feel "normal" to the vast majority of the audience. Cheers! --Paul XHTML Template Architecture Zope should increase usability and promote separation of tiers by moving to a presentation and data architecture. Site Designers will author and round-trip presentation objects in familiar tools. Combined with the Document Architecture, this proposal establishes the fundamental content and presentation architecture for Zope. Note on status The great folks at Hiperglocia have already produced something quite similar to what is discussed here. Their Zope product is called "HiperDOM", http://www.zope.org/Members/lalo/HiperDOM/. Note On Jargon The choice of term for the presentation object has been contentious. Right now the list of choices include: template, view, page, or stylesheet. This proposal doesn't make the decision on the jargon. Rather, the tier is usually refered to as the presentation. When a choice has to be made, such as the Architecture section, Template is used as the temporary choice. Problem Zope currently has a model of DTML Documents and DTML Methods as the basic data and presentation tiers. However, this model is confusing, doesn't promote separation of tiers, and doesn't work with popular web design/authoring tools. For the presentation tier, Site Designers are faced with an idiom in which well-formed HTML is split between two "files" (standard_html_header and standard_html_footer). Also, the presentation mockups are "taken away" by programmers who insert a bunch of alien tags that don't present anything on the screen of the web design tool. Worse, DTML is fundamentally impossible to edit directly by classic tools due to the "source vs. rendered" issue. That is, when a Site Designer wants to edit a piece of DTML, they will get the *rendered* version of the DTML for editing, where all the DTML tags are expanded into HTML. In summary, there is a tremendous usability issue, both conceptual and in practice. There are additional problems: o DTML is considered a "proprietary" language o Data needs to have different presentations used under different conditions o Consulting practices need a way for customers to start seeing screen results immediately without having the programmer "steal" the templates by converting them to something alien Goals The goals of the Presentation Architecture are: o Clear separation of presentation, data, and logic with clear concepts in Zope for these tiers and audiences o Increase usability by allowing round trip presentation with common tools and familiar concepts o Allow those presentation tools to work by having well-formed markup (e.g. no separation into header and footer) o Supplant "proprietary" markup language (DTML) in presentation tier by leveraging standards (XHTML) (Note: "eliminate" isn't the target as it might not be feasible in 100% of the cases) o Match the appropriate level of capability currently in DTML usage o Simple presentation reuse within reach of standard tools Solution Zope should move to an architecture with clear separation of presentation, data, and logic, with clear concepts in Zope for these tiers and audiences. The presentation tier will get a tremendous usability increase by allowing round trip presentation with common tools. This also ensures that Site Designers finish with the same stuff they started with, meaning the programmers don't come in and cast their work into "code". The architecture should make sure those presentation tools are effective by having well-formed markup (e.g. no separation into header and footer). Zope should supplant "proprietary" markup language (DTML) in presentation tier by leveraging standards (XHTML). It might not be possible to 100% remove DTML in 100% of the cases, but it should be a rarity instead of the rule. The new presentation architecture should match the appropriate level of capability currently in DTML usage. For instance, common idioms like browse-by-batch should be possible. The new presentation architecture should leverage this new XHTML approach, XSLT, and perhaps DTML. Site Designers want simple reuse of content within reach of standard tools. For instance, the Site Designer wants a standard copyright "asset" on the bottom of every page. The presentation architecture should not know about the details of the data tier or the logic tier. Rather, the presentation architecture will just provide a view onto the model expressed in the object system, where the model uses standard APIs. A **page** is the result of applying presentation to data in the object system. A page is a particular result of a URL when viewed under certain conditions. Ultimately, data such as documents will be matched to a presentation according to implicit and explicit rules. The simple case will be simple (invisible); the complex, custom cases will be straightforward. In the initial rev of the architecture, only one presentation will be implicit for a resource. Other presentations of that resource will require explicit traversal to obtain them. Architecture The Presentation Architecture initiative will deprecate the presentation aspects of DTML Document and DTML Method objects. Instead, a new kind of object, code-named Template, will be used. Templates have two varieties: XHTML Templates for Site Designers that use HTML layout tools, and XSLT Templates for XML programmers requiring high-end processing control. Site Designers will create XHTML Template objects that control look-and-feel across numerous content resources in a site. The Template object will allow multiple formats. XSLT will be used for high-end presentation oriented towards XML programmers. DTML might be retained as a template format for certain presentation tasks. However, the focus of the new Template architecture will be a new format based on XHTML. This XHTML Template will be focused on giving Site Designers round trip presentation authoring using the dominant tools such as Dreamweaver. XHTML Templates will be complete, well-formed HTML that look like a resource will look when the XHTML Template is applied to it. There will be no "source vs. rendered" issue. In this Presentation Architecture, Site Designers can start mocking up a Zope site by editing real HTML and graphics in Zope from the very beginning. There will be nothing Zope-ish in what they are doing. As the mockups turn into live content, certain regions of the XHTML Template will be "connected" to the live content. This is done by indicating the object data that will be inserted when the XHTML Template is applied to content. However, there will still be sample content in the XHTML Template that lets the presentation task continue. This sample content gets discarded when the XHTML Template is applied to a piece of content such as a Document; the connected regions in the Document replace the sample content in the XHTML Template. These connectors will exhibit themselves by simply adding Zope attributes to existing, valid HTML tags. These attributes will follow the XHTML extension facilities of namespaces and will be silently preserved by authoring tools. The attributes indicate that this region of the XHTML Template will be replaced by content from the Document being viewed and from other content in the object system. The connectors will allow different "modes" that provide the alternation and conditional facilities present in DTML. These modes are just simple attributes to extend the semantics of existing XHTML tags. The XHTML Template architecture will facilitate re-use by allowing blocks in the XHTML Template to be named as a (component/library/asset) block. As the Site Designer authors and updates the block of XHTML, Zope saves the named block as a separate object in Zope. This named object can then be used in other XHTML Templates. Internally, XHTML Templates will use the standard XML parser in Zope instead of maintaining a separate parser. It is possible that the XHTML Template will be broken into a DOM and stored in the ZODB as separate database objects for different nodes. Scope Defining the document architecture has been moved to a separate proposal. XHTML Templates will be focused on the presentation needs and skill level of Site Designers. Highly complex presentation tasks that blur into programming tasks will be left to the domain of XSLT Templates. Thus, the XHTML presentation format is scoped by focusing on usability for the Site Designer, non-programmer audience. XSLT Templates will be aimed at XML professionals and at highly-complex presentation layout needs. The XHTML Template will not do transforms. If a Site Designer needs to enforce any rules on markup in content, some other step will have to be taken to ensure the Site Designer is handed acceptible, "stripped" content. DTML has permitted the rendering on non-markup such as SMTP messages. The XHTML Template does not try to satisfy this job. Other objects, such as DTML Methods or Python Methods, would have to handle this chore. Clarifying the "DTML namespace" confusion felt by authors is not in the scope of this proposal. While providing and documenting integration with a few important tools will be done, this project won't integrate every tool. Later projects will deal with the rules that allow implicit binding of presentations to resources. Risks *Role of DTML*. What is the role, if any, of DTML in the future? *Replacing magic with magic*. If too much magical stuff happens (e.g. how data gets implicitly bound in different ways to presentation, the component idea creating new stuff in the background) for people to handle, we're just replacing the confusing old magic with the confusing new magic. Deliverables The first deliverable will be rich user documentation. This will supplement use cases. The second deliverable will be a documented API and explanatory text to go along with the API. The third deliverable will be an alpha-quality implementation on a branch.