[ZODB-Dev] CMS on top of ZODB

[Dieter Maurer]

The company I am working for used S=F6rman-CMS (formerly POET-CMS),
an SGML/XML Content Management System on top of the POET object database.
Unfortunately, S=F6rman Germany went bankrupt and
we do not trust the company that took over the CMS to fulfill our
support requirements (as it took over only the software but no people
involved with its development). Therefore, I am exploring the possibility
to implement our core requirements on top of ZODB.

Core requirements are:

  *  management of many (in the order of hundreds of thousands) SGML/XML
     documents, some of them in the order of several MB in size

  *  documents are made up of components with versions (called
     "history" in Zope/ZODB) which can be shared between
     documents

  *  highly concurrent use (about 200 concurrent active users with
     write access; steadily increasing)

  *  stress read access during production (of publications produced
     out of the CMS)

  *  X-Metal integration (on top of an intranet protocol).

  *  High availability/replication


I think I will be able to implement about 90 percent of these core
requirements on top of ZODB in a matter of weeks.

However, there are some tricky issues, I would like to discuss:

  *  ZODB's optimistic concurrency control

     It requires that the probability of conflicts is low (fullfilled
     in our setup) and the effect of conflicts is not desastrous
     (not fullfilled in our setup; checkins can take many minutes
     and it were bad when they had to be retried due to conflicts).

     If necessary (after having exhausted application specific conflict
     resolution), I would implement (object) locks in the ZODB:

       Read locks would do nothing except prevent to acquire a write lock
       (by a different connection) and to write the object.

       Write locks would prevent to acquire other locks on the object
       and behave like a simulated write (with identical content)
       with respect to cache control.


  *  Versioning/History

     ZODB's version/history support is almost identical to that implement=
ed
     by S=F6rman-CMS. Its only drawback: it gives us only very limited
     control over the versions.

     I think, we could enhance the packing process to recognize
     special objects (based on classname) to hold references
     to older versions. We could then use a standard marking
     garbage collection for packing: current versions and
     versions referenced by (current) special objects are
     in use; any version referenced by a version in use is in
     use.

     The "version referenced by a version" is a bit tricky, as
     a version references another version only indirectly:
     it directly references an object and it is associated with a time;
     it references the object's version that was current at this time.

     Of course, packing would become considerably more expensive...


What do you think about the prospects of these extensions?


I can make extensions to ZODB (with respect to programming skills
and employment contract obligations) and donate them back to the core,
when you are interested...


Dieter