[ZDP] Feast your eyes on even more Documentation!

[Michel Pelletier]

Note:  This is an chapter from 'The Zen of Zope'.  This is posted for peer
review.  This is slightly older material, so corrections, please.


-Michel



Zope Internals

This section is about the Internal workings of Zope, the Objects that
Zope defines and uses, and the Application Program Interface that
programmers can use to extend Zope by writing applications (called
'Products' in the Zope world). In our examples we create an actual
Zope product that other applications can be modeled against. 

Internally, Zope is looked at two ways, as an Application, and as an
Application Framework. The line between these two views is fuzzy,
but there is some distinction in Zope's source code. As an application,
Zope's job is to present and manage an object database through the
web. As an application framework, Zope provides objects and methods
to help programmers create their own applications to work with the
web and the database in new or specific ways.

To assist the programmer, Digital Creations defines an Application Program
Interface
(API) to Zope. Zope defines a variety of objects (many of which have
been used so far, like Documents, Folders, etc.) for the programmer
to reuse and extend. Programmers also define new objects, or 'convert'
their existing objects to work with the Zope API. Because all of this
is written in Python, extending and reusing components is very easy.
It is assumed throughout this section that the reader has some familiarity
with Python and it's concepts; there will be no Python tutorial. Please
see the excellent tutorial at the Python web site, (http://www.python.org/). 

The key to Zope enlightenment comes from understanding its internals.
Since Zope is open source technology, its Python source code can be
studied and learned from. Zope is a rare type of program; most programs
that have powerful features are based on lots of small complex components,
each one built separately to do separate things and all interrelated
in loose, possibly random ways depending on the use of the program.
Sometimes this is the only way to develop certain applications (like
certain popular Operating Systems). Zope is different in it's design,
it is based on a small collection of very powerful, very clever components.
Much of their cleverness comes hand-me-down style from Python, and
much of it comes from some special engineering concepts pioneered
by Digital Creations.

When using a CGI based web server, the next layer 'above' the web server
is the PCGI component of Zope. PCGI is a wafer-thin piece of C code
whose innards will not be discussed in too much detail here. When
using Medusa or ZopeHTTPServer this layer is dispensed with, and we
move on to the next layer of Zope and the core of it's Zen, the
ZPublisher.Publisher
module. 

0.1 URL/Object Transversal

The heart of Zope is the ZPublisher.Publisher module. This, for historical
reasons, is sometimes referred to as Bobo. The Publisher takes user
requests and translates them into object references in Python. The
Publisher is like a gateway from the web-with it's request/response
event driven model-into a Python object hierarchy. Python applications
can be built in a straightforward manner this way, breaking down the
problem space into a hierarchy of objects.

The Publisher's main argument is the Module to be published. When a
request comes in, the Publisher transverses the subobjects of the
published Module. For example, the URL 'http://.../x' looks for a
sub-object named x in the published Module and returns it to the requester.
The Publisher also marshals data in the request into arguments for
the method called. 

Originally Digital Creations released this software as Bobo under an open source
license. Bobo was the heart of a variety of other open source components:
Document Templates, Bobo Call Interface, and Bobo Persistent Object
System (BoboPOS). Digicool used these components to build Principia,
the predecessor to Zope. When Digicool decided to release Principia
as open source and change it's name to Zope (after much difficult
consideration ie. finding a domain name that wasn't taken), all of
the individual component modules were renamed. Principia became Zope,
Bobo became ZPublisher.Publisher, Document Templates became ZTemplates,
Bobo Call Interface became ZPublisher.Client, and BoboPOS became the
Z Object DataBase. 

So to old-hat Bobo programmers Zope is one big Bobo application. Since
Zope builds its foundation on ZPublisher.Publisher understanding it
is the first crucial step toward understanding higher Zope Zen. In
this section, ZPublisher.Publisher and Bobo are interchangeable.

More, explain the details of traversal in Pythonian terms, show a simple
Bobo app.

0.2 Acquisition

0.2.1 Implicit

Acquisition was used earlier in the DTML section. It was used by Documents
that wanted to acquire attributes, like External Methods, from the
object that contained them. This happend without needing to call any
special functions, the DTML simply referenced the attribute as if
it were an attribute of itself. This ability to acquire attributes
implicitly, is the magic of Acquisition.

Acquisition is simple and very powerful. Simply stated,
an object can acquire methods and attributes from objects that contain
them, at run time.

Acquisition is sometimes confused with Inheritance. Both Acquisition
and Inheritance support the concept of factoring functionality into
hierarchies, but they work quite differently. A Python class can inherit
the attributes of a parent class, similarly, a Python object can acquire
the attributes of a parent object. Think of designing a car. The car
itself would be defined by a car class, and the doors would be defined
by a door class. As a sensible car designer, you would want the door
to have the same color as the car. It would be silly for the door
class to inherit the car class, why would a door need all that functionality
like engines and transmissions and gas tanks? It would make more sense
for the door object to acquire whatever color its parent car object
is. Here is a simple Python example:

import ExtensionClass, Acquisition

class bc(ExtensionClass.Base): 

    color='red'

class ac(Acquisition.Implicit): 

    def iscolor(self): 

        return self.color

>>> a=ac() 

>>> b=bc()

Now we have created two instances of class ac and bc, objects a and
b. a inherits the object class ExtensionClass.Base. b inherits
Acquisition.Implicit,
the main class that defines Acquisition behavior. 

Even new Python programmers will spot some obvious strangeness to the
two classes defined. Class ac defines a string attribute, but doesn't
really do anything with it, and class bc defines a method, that references
an attribute that doesn't even exist in that object! Calling:

>>>b.iscolor()

causes the Python interpreter to complain about not finding the attribute
color. So how can the iscolor method possibly be of use to us? 

>>>a.b=b 

>>>a.b.iscolor() 

'red'

What happened? First, b is assigned as an attribute of a. a now contains
b. b's iscolor method is accessed in the context of a. Since b has
no attribute color, the acquisition mechanism kicks in and searches
the object which it has been called through, object a. Thus, object
b acquires, at run time, attributes from the context in which its
iscolor method is accessed. This little magic is done with Acquisition
wrappers. When an Acquisition.Implicit object is used in the context
of an ExtensionClass.Base object, an Acquisition wrapper is returned.
In the above example, a.b returns an acquisition wrapper than contains
references to b and a. It is this wrapper that implicitly check the
a object for attributes not found in the b object. 

Returning a wrapper around an Acquiring pair of objects is another
little piece of trickery that throws off beginners. In the code above
a.b is b evaluates as false, because a.b returns a wrapper around
b, not b itself. To get to b, use a.b.aq_self.

>>>a.b.aq_self is b 

1

The extending object, a, can be access with aq_parent; a.b.aq_parent
is a evaluates as True. The obvious question is: "Why access the a
object through aq_self? Why not just use a?". The answer is the name
of a is not allways known. This question has a loose analogy in the
C programming languages, where beginners often ask the question: "Why
reference variable a through a pointer? Why not just use the varable
name?" Anyone who has tried to teach this concept in C can appreciate
the power and confusion these kinds of language constructs create. 

It is possible for aq_self to be an acquisition wrapper, and wrappers
may nest to any depth. The aq_base attribute peels away all the wrapper
layers and returns the a bare unwrapped object. 

more about aq_base...

0.2.2 Explicit

An acquiring object can acquire any attributes from a base class, except
those that begin with an underscore '_'. This is a primitive access
control method which hides private attributes in base classes from
acquisition.

The above examples explain Implicit acquisition. There is another method,
Explicit Acquisition. To use it, let's redefine the class ac defined
in the above example:

class ac(Acquisition.Explicit): 

    name=Acquisition.Acquired 

    def iscolor(self): 

        return.name

Although name was the only attribute to be defined in the class bc
to begin with, Implicit acquisition would have acquired any others
defined. This many not be wanted, only one or two attributes from
a containing object may be desired. If that object has many attributes,
problems with namespace collision arise. Perhaps the programmer doesn't
want the object to have a certain attribute, or the object may want
to create that attribute sometime later if it doesn't exist, or the
attribute in the container object may have a name that conflicts with
the overall design philosophy of the contained object, and might confuse
other programmers. In the new ac class, no matter how many more attributes
are added to a containing object, only name is acquired. Explicitly
acquiring attributes in this way overrides the underscore '_' security
mechanism, and containing object attributes that begin with underscores
can be explicitly acquired with _hiddenattr=Acquisition.Acquired.
An example of how Zope uses this mechanism is the security management
screen for Documents and Folders. One option is to acquire those settings
from the object that contains the object in question. These attributes
however begin with an underscore so that they cannot be overridden
in DTML, which aquires attributes implicitly. 

So far, we've used acquisition from the perspective of the base class
object. In our examples, we've been typing at the level of the interpreter,
and all accesses to acquiring objects has been through the acquired
object. Programmers are not allways in this position. 

Acquisition defines a special method, __of__. So far we've seen the
acquisition object work from the perspective of the base object. As
long as attributes of contained object are referenced through the
base (container) object then acquisition works. What happens when
we want to start from the perspective of the acquiring (contained)
object? This is where __of__ comes in handy. 

>>>x.y.isname() 'red'

is an example of the first perspective.

>>>y.__of__(x).isname() 'red'

is an example of the second. 

(more on the mysterious __of__())

0.3 Object model

Zope follows a design philosophy. Now, lots of people have different
ideas of how things should be done, and there are some programmers
who prefer writing their own management interface, authentication
system, etc. Perhaps they only want to implement one of these subsystems
themselves. Zope does not restrict programmers to stick to the 'hard
and fast' API because there is no such thing. Remember, Zope is layered.
It's components work quite well together, and they work just fine
apart. 

Third parties who wish to extend Zope, or write an application based
on the Zope framework, write Products. Products are Python packages, and their
details can be further explored on
the Python website. Zope packages must have at lest one __init__.py
file in them, and the file must conform to the Package initialization
protocol with Zope.

A Product usually defines an Object that Zope can deal with. It is
not necessary for products to be Zope objects, or for them to conform
with Zope's standards, but they must conform with some protocols,
and obviously not do anything harmful to the Zope system.Products
are a hook to extend Zope, and there are not any hard and fast rules
to how this extending can be done. It is encouraged to follow along
with the style, and to diverge only when truly better results will
occur. There is obviously not point in confusing users with wildly
different interfaces between components in the same system.

There are two basic building blocks of Zope. The Folder, and the Item.
Folders are containers, and contain Items or sub-Folders. Item are
commonly extended into simple 'widget' style Products, an object that
does one specific task and doesn't need to contain or manage other
objects. Folders are often extended into larger applications which
must contain and manage sub-objects which may be custom designed or
standard Zope objects. In application programming, Folders are handy
for wrapping up a whole application as a collection of customized
Item-style widgets. This technique will be further explored in later
sections.

All of the Zope objects used so far are based on either Items or Folders.
Documents, Files, Images, and Mailhosts, are all examples of Item
object. Zope comes with two common Folder objects: the Folder and
the User Folder.

There are other objects that come from Zope that provide specific Zope
services. The Item and Folder object provide a means of constructing
trees of objects, but this is not all there is to Zope. Other mechanisms
work behind the scenes. In the example below, an Item object is inherited,
but a Folder object could be used just as well. Complete example of
Item and Folder products are given in sections x and y.

import OFS.SimpleItem 

class X(OFS.SimpleItem.Item): pass

Now this object can be handled by Zope. However, wouldn't it be nice
to be able to acquire some things from the Folder object that will
eventually contain this Item? Well in that case, let's add Acquisition.Implicit.

import Acquisition 

class X(OFS.SimpleItem.Item, 

    Acquisition.Implicit): 

    pass

Now the object can acquire attributes from the folder that contains
it. This is handy so the widget can refer to any standard attributes,
like standard_html_header. Also, if writing a Folderish application
that contained a lot of customized Item widgets, common objects can
be shared among the widgets using Acquisition.

So now the object is ready to be stuffed into Zope. But don't go stuffing
yet. More features, like ensuring the object gets saved in the Zope
database, and some User restrictions and security, would be nice too.
That bring us to the next section. 

0.4 Object Database

The Z Object Database is a core part of Zope. The Object Database defines
a Persistent Object System (POS). This allows applications to be written
without having to continually save state information in files. Objects
are Persistent which means they live long and happy lives, looking
like object in memory but having the courtesy to save themselves automagicly
in the POS database whenever they change. In order for Zope to stuff
an object in the database though, that object must first inherit the
hooks and handles that the POS uses to manage the object. These can
be added to our X class: 

import BoboPOS.Persistence 

class X( OFS.SimpleItem.Item, 

    Acquisition.Implicit, 

    BoboPOS.Persistence.Persistent ): 

    pass

The BoboPOS.Persistence.Persistent class is all that needs to be added
to the object to make it stashable and cacheable by the POS. This
is a part of Zope that rarely has to be messed with or extended, and
is thus the simplest component to add to the class.

It should be noted that in previous versions of Zope, when it was called
Principia, the Persistence.Persistent class was not contained in the
BoboPOS package, but was a standalone module. For applications to
be backwards compatible with older Principia versions of Zope, the
Persistence.Persistent class must be imported with care:

try: 

    import BoboPOS.Persistence 

    Persistence = BoboPOS.Persistence 

except: 

    import Persistence

and then subclass Persistence.Persistent without BoboPOS prepended
to it.

I think this might be wrong, I seem to remember BoboPOS allways coming
in Package form, and me installing wrong...

0.5 AccessControl: Users and Roles

So now instances of the X class can act like Items in Zope, acquire
attributes from their containers, and be stored in the POS database.
Zope provides a prebuilt security mechanism for the programmer to
control what Zope users can have access to various attributes of the
object. If we want to X class to inherit this mechanism we must extend
it further by inheriting the AccessControl.Role.RoleManager class: 

import AccessControl.Role 

class X( OFS.SimpleItem.Item, 

    Acquisition.Implicit, 

    BoboPOS.Persistence.Persistent, 

    AccessControl.Role.RoleManager ): 

    pass

Zope's security system is based on Users, Roles, and Permissions. All
Zope objects can acquire the security settings of their container,
thus maintaining the ability to work in a management delegated environment.
As an example, a top level manager can create a dozen folders, and
in each folder put a user folder. In each of these user folders their
can be one user with 'Manager' privileges. Now any sub-managers can
manage their folder (which to them appears to beacquire the top level),
and any subobjects that get created implicitly acquire the security
settings of their containing folder. Because sub-managers are not
defined in the top level folder, only in the sub folders, the top
level will never be available to them, but any object they create
under their folder acquire the setting to be under their control.

(more on access control)