[Zope-PTK] Best practices for Zope development, staging, and production

[Joachim Werner]

Hi!

We at iuveno face similar problems.

> The Problem
> 
> How do we develop, and keep all the client websites working all the
> time, while maximizing reuse; i.e., having them benefit from upgrades
> done by different parties (in this case us and DC) at different levels?

This is not the perfect answer to your problem, but it might help if you want
"zero downtime":

We are thinking of using a cluster of machines to run Zope on, with a load
balancer and proxy as a front-end. In such a scenario you can upgrade your
customers' instances one-by-one, test them for any problems the new base
classes might cause (for this you'll have to use a backup of the data.fs, not
the production one ...) and then just switch over to the new one.

> Currently I don't see an easy way to separate
> presentation, logic and content without giving up working in Zope
> objects and putting all the content in a SQL database.  Does anyone have
> experience with this?  How is Zope.org updated? 

It's perfectly possible to have classes that only contain data and won't break
or get lost when you update the presentation logic or look and feel. Saying
this, I don't think that all Zope sites do this consistently.

What we are doing (or planning to do) is the following:

We have a basic toolkit (combining the PTK and "ExoPortal") layers of your
model as we'll probably not use the "out-of-the box PTK". This toolkit can be a
filesystem-based product (right now the prototype is a ZClass package). The
toolkit provides the basic functionality for the sites and provides hooks for
customizations. E.g. a data object may be displayed using the standard "view"
method, but if it brings its own "view" with it the standard one is overridden.
All this is being handled by acquisition magic.

The clients' organisation structure is plugged in as ZClass products. E.g.,
for our prototype toolkit we have created sample object sets for a university
intranet and a corporate website (with objects like "location", "administrative
unit", "manager", "employee", etc.). ALL the data is kept there (with the
exception of session data or in some cases the user credentials, that might be
supplied by an LDAP server). Nothing will break if you update the underlying
ZClasses' method objects, as long as they don't get different names.
Zope/Python stores just the plain data in the instances of the ZClasses, so
when they are called they should work fine with the new ZClass versions. If the
standard methods provided by the toolkit layer or by Zope itself don't do the
job, these objects can provide their own methods.

We have not explored the ZPatterns framework yet, but it seems to be very useful
here, because it adds a clean abstraction layer. So your data object sources
can just be plugged in, making it possible to import data from non-Zope sources
like an RDBMS or an LDAP server.

The look and feel is again separate from the other components. It's
currently defined by a chrome folder that contains just images and style
sheets, but it could also include different basic layout templates (maybe using
XML or the HiperDOM approach).

This will only work perfectly if you want to serve HTML. As soon as the same
data sets and application logic are also used for other protocols like WAP or
XML you'll probably have to change the approach slightly. We are thinking of
only providing pure XML on the lower levels and then rendering the output using
different DTML methods for the different protocols or later XSLT.

An example to illustrate what I'm taking about:

We want to display a portal site for a project. It normally consists of a list
of all the project members, a status box, project news and a calendar.

Data objects will hold all the necessary information, like the project members,
news items etc.

Filters (e.g. a ZCatalog) will provide the lists of objects customized for the
project portal (i.e. only the members of THAT project, the news that are
specific or relevant to THAT project etc.).

All this will be rendered as a single XML document containing all the
information.

If the client supports XML/XSLT natively, it might be provided with this "raw"
document and render it using the XSLT style sheet.

If the client supports plain old HTML, the page is rendered as an HTML page
using HTML tables. If a list (of news items or members) is too long for display,
batches of 20 or so are displayed instead.

If the client is a WAP phone, you'd separate the XML document's content into a
deck of WML cards providing a basic navigation and separate pages for all the
elements like news or members, with batching of 3-5 maximum.

Except for the presentation layer, the site would not have to be changed at
all if a new type of client is implemented.

If you keep the very basic implementation of your own "API" relatively stable I
don't think your system will be too complicated to update. In a well-designed
object architecture the objects are more or less self-contained.

BTW: Using SQL doesn't help at all. It even causes additional problems: If you
change the data sets of your objects in a way that is incompatible with the
data you would have to change the SQL table structure as well. So that doesn't
buy you anything. If you use objects, e.g. ZClasses, instead, you can at least
apply minor changes like adding a new property to the objects without breaking
anything. And you don't have to do it twice -- in the SQL AND the object.

Even if you have objects in your database that are incompatible with your new
approach you might easily write a DTML or Python script that converts them to
the new class' objects.

Your main problem seems to be the two layers you get by using the PTK and an
own customization layer on top. If an interface changes in the PTK you have to
rewrite stuff in the ExoPortal layer, but that again should be easy if you use
a clean object-oriented approach (e.g. implementing all the general things in
base classes to only have to change them once in the base class).

One additional note on data storage: If you want to separate your data and
logic physically, e.g. to back up the data regularly without always having to
back up all the application logic code that hasn't changed with it, it should be
possible to use different mounted ZODBs that then can be backed up separately. 

Joachim.

__________________

Iuveno - Smart Communication


Joachim Werner


_________________________________________

Marie-Curie-Straße 6
85055 Ingolstadt

Tel.: +49 841/90 14-325 (Fax -322)
Mobil: +49 179/39 60 327
E-Mail: joachim.werner@iuveno.de/joachim.werner@iuveno-net.de
WWW: www.iuveno.de/www.iuveno-net.de