[Zope] ZEO app architecture options

[Michel Pelletier]

Jimmie Houchin wrote:
> 
> Hello,
> 
> Can I architect a website like this using different Zope Storage Servers
> for the various directories or 'would be' tables in an RDBMS? Yes, I
> understand the impact of the OS filesystem concerning ZODB size.

ZEO is transparent to the application, it is a _singular storage_
meaning that all the issues of multiple databases still apply, re, they
are not yet supported.

> It would be nice to be able to partition the database into
> stable/volatile files. Since the stable partition is also the largest,
> it seems this would also help performance of the more dynamic section.

What this needs is the ZDBF (Zope DataBase Framework).  This is a layer
that sits above ZODB.  Currently, ZODB takes care of all these issues
(or punts on them) when really a higher level abstraction should handle
multiple database issues (including any future extensions to the
Relational/Object model).  Obviously, this is a big task.  See paul's
email about resources. ;)

> Also, when comments are made about Zope's ZODB or ZSS not being
> appropriate for heavy write apps, does this refer to apps doing
> "updates" on objects or doing "inserts" or both using SQL terminology?

First, keep in mind that to a certain degree, all databases are
sensitive to the high-write issue.  ZODB is more sensitive because it's
conflict resolution is optimisitic.  FileStorage is also sensitive
because of the  inherent behavior of filesystems, and ZEO is sensitive
because for every object written, an invalidation message must be sent
to all other ZEOs so that they are given the chance to invalidate the
object in their cache.  This also increases chances of collisions
because the Time To Write a record is extended for varous reasons
(network delay, invalidation delay, etc...).
 
> Would structuring an app like above help in write situations?

Not without ZDBF.  Multiple database support would allow you to use
different, less write sensitive storages like a RelationalStorage or a
BerkeleyStorage.
 
> I am trying to determine the desirability or necessity of an RDBMS for
> my idea. I like keeping it in Zope if reasonable and ZEO can definitely
> help. If need be I'll go to an RDBMS for certain data.

Do some benchmarks, ZODB is very fast if you are a little more clever
about how you change your objects.  If you need more write intensiveness
go relational.

-Michel