[Zope3-Users] zalchemy integration

[Jeff Shell]

On 8/10/06, Carlo Cardelli <ccardelli at iv-srl.it> wrote:
> Once skipped this, the process halted while adding the 'blank' object to
> the db: in zope.app.container.browser.adding, in the method add(), the
> following lines failed:
>
> <code>
>          container[name] = content
>          ... one ininfluent-for-this-problem line ...
>          return container[name]
> </code>
>
> Here, the last line failed because the SQLAlchemyContainer completely
> disregards the "name" attribute (as specified in container.txt), so
> retrieving data using the same name is not possible.

Hm. It seems like a NameChooser should *somehow* be giving you a
usable name. That pair::

    container[name] = content
    return container[name]

seems redundant on first look, but it's what gives a bound object in
typical Zope 3 situations. However, this is not a typical Zope 3
situation. Well, it actually brings up the point with which I had so
much frustration a few months ago: what is a typical Zope 3 situation?

I'll elaborate on my experiences with SQLAlchemy + Zope in a moment.

> Maybe I could spend some time in it, but this "basic" problem leads me
> to some doubt about the state of the zalchemy package and its
> integration with Zope3.
>
> In the mailing list archives I found some discussion about ORM problems
> (2-phase commit, thread support and so on) but nothing specific about
> zalchemy, and (as far as I can see) no "definite" solution on the
> sqlalchemy integration.

"Object Relational Mapping is the Vietnam of Computer Science"
http://blogs.tedneward.com/2006/06/26/The+Vietnam+Of+Computer+Science.aspx

:)

Part of this depends on what you want with integration. Personally, I
stay away from the ZMI as much as possible. I also stay away from
`zope.app.container.browser.adding` as it's just... strange. Too ZMI
specific, if I remember correctly.

> Could someone tell the current state-of-the-art about this, or his/her
> experience about integrating sqlalchemy in other ways?

I don't know about state of the art, as we haven't done any stress
testing on our system yet, but we've integrated sqlalchemy in other
ways.

I took inspiration from ZAlchemy's branch that supported SQLAlchemy
0.2. I don't know if this is in the trunk or not. That's our core
bridge layer. Similar to ZAlchemy there's a simple utility that is
responsible for maintaining the database connection. Working with
SQLAlchemy 0.2 is a much easier than 0.1 with its support for dynamic
metadata and its stepping away from its own thread-local management of
sessions.

Unlike ZAlchemy, we don't use ZCML to register tables or classes or
anything like that. The only ZCML we add is something to the database
connection and bind it to a SQLAlchemy `DynamicMetaData` object in our
application.

Classes and mappers are established via a common `storage.Base` class
which uses a metaclass to build the mapper and relationships. Most of
the relationships are defined in Zope Interfaces using some custom
schema fields, and are bound to the classes by Descriptor (Property)
classes.  These are responsible for doing the Zope `location` binding
so that traversal can happen.

We have two property types for 'many' relationships. `Many`, which
acts like a list; and `Container` which acts like, um, a Zope
container. I doubt these would ever work in the ZMI, but they might::

    def __setitem__(self, name, object):
        """
        It is assumed / hoped / foolishly assumed at this point that the
        'name' is pointless here, since it should already be bound to the
        object. Use 'add' instead.
        """
        self.add(object)

heh. We bind `__name__` on our objects to a string representation of
the primary key, and the code above makes the assumption that it's
already set. So we discard it. But then again, we don't play the
`container[name] = obj; return container[name]` game. The
implementation of `Container` is primarily for traversal so that::

    promoters/1/contacts/3/addresses/2/@@edit

can work in a URL.

I can't go into details about our system. It's intertwined with other
libraries and frameworks that my company has built up over time, none
of them releasable at this time.  That said, we've gotten pretty good
results so far. However, we're also massively behind schedule. This
project we've been working on has involved a lot more engineering than
we foresaw; but the SQLAlchemy bridge has generally worked out. What's
worked for us:

Using metaclasses and a strong Base class for the SQLAlchemy stored
objects. This Base class has a lot of class methods for querying and
creation, and implements ILocation. This Base class also provides the
tools to ensure that the SQLAlchemy Session is joined, etc.

The base / metaclass provides support for doing some declarative work
at the class level for connecting Classes to Tables, and for setting
up extended properties. No ZCML required, allowing non-Zope programs
to be able to at least use our Model classes if they have the
libraries imported and establish the database connection.

At the root, we have 'ModelControllers' which are bound to our Site
objects (the few things stored in the ZODB) via Zope's adapters. These
provide the base traversal points. They implement a read container
interface, but no writes. As such, our add forms are responsible for
the full creation. Sometimes it's as simple as this, using `formlib`
::

    class TagAddForm(layout.Form):
        label = "Add New Tag"

        form_fields = form.Fields(ITagSchema)

        @form.action("Add Tag")
        def handleAdd(self, action, data):
            tag = Tag.create(**data)
            ruts.event.created(tag)
            self.flash("Tag '%s' Created" % data['title'])
            return self.redirect(self.backObject())

        def backObject(self):
            return self.context

The other benefit, some of which is seen here, is building up a bit of
our own web framework, providing features that are useful to us while
avoiding a lot of the Zope 3 elements (many in `zope.app`, at least in
Zope 3.2) that we don't need. Our most used `zope.app` imports are
ViewPageTemplateFile and zope.app.form widgets.

The main thing is just trying not to take on too many of the
expectations that Zope may have about implementation. This is actually
pretty easy since Zope 3 is so interface based. The downside is that
some of the built-in automatic tools, such as those in/for the ZMI,
may not work as expected. Personally I see the ZMI as a limited road,
useful for managing local utilities (caches, session data containers,
etc), but not a good application environment. We don't use
INameChooser, ISized, etc, etc, because our application has no need
for them. A few months ago I was crying for a Zope package that was
devoid of many of these items. They're useful items for certain kinds
of applications, but they can be a horrible distraction for others. I
think that my company's recent experiences with Zope and SQLAlchemy
show that the fundamentals of Zope can be terrific toolkit for rich
object oriented RDBMS / Business Object backed applications that have
nothing to do with content management.

So yes, it is possible to have good SQLAlchemy integration. But
'integration' may mean different things to different people. Some may
want invisible or near invisible integration with conventional Zope
content-management-esque concepts, integration with the ZMI, the
dublin core, etc. That's overkill for my needs, which is why I've
stayed away from those implementations. Having Location, Security,
Adapter binding (which yields views and URL traversal) is just about
what we need. The rest we provide through our own business logic as
Views, Utilities, and plain old Python classes and functions. And only
one new ZCML directive (yay!).

-- 
Jeff Shell