[ZODB-Dev] Advice needed
Andreas Jung
lists at zopyx.com
Sat Jun 24 03:24:37 EDT 2006
--On 24. Juni 2006 08:53:43 +0200 Roché Compaan
<roche at upfrontsystems.co.za> wrote:
> I am curious what other strategies besides QueueCatalog you employ? Do
> you ever use multiple backends for your apps? How do you decide that
> this data belongs in a relational backend? How structured must the data
> be, or how many records must be written how often?
>
> I find most data is highly structured (fixed schema), but this doesn't
> make me choose an RDMBS - the frequency of writes, concurrency and
> record volume does.
It is often the case where you have more complex data models that really
require a RDBMS. It is often the case that you have to perform complex
queries on your data. The ZCatalog is often just too weak and too slow for
such apps. Our CMS (to which Dieter often refers to) uses the ZODB to store
large amounts of SGML data. Since the documents fits perfectly inside a
hierarchy the ZODB is the perfect choice. However we have some apps around
the CMS that provide additional functionality or use the CMS for a
different purpose. Some of these apps use the content from the ZODB but
store their metadata in postgres. One not unimportant advantage is that
other ppl can run their own reports etc. using a postgres client without
approaching me to write a script or something in Python to get requested
data out of Zope.
One particular app that I have been working on uses very complex queries
with lots of join etc....it would be hard model to implement such queries
on top of the ZODB/ZCatalog. Another point is performance: this app often
has to perform a lot of insert/update/delete operations within one
transaction (up to 1000 modifications). Postgres takes perhaps 5 seconds
for such a complex operation. You will never reach that performance with
Zope...touching 1000 objects and reindexing them will take much longer
(unless you adjust your data modell for performance reasons to the ZODB
needs).
So what I want to say is: the bullheaded idea to stick everything into the
ZODB or into a RDBMS is just stupid. Smart people think about the data
storage before starting a project. That also reminds me of some postings on
the Plone mailinglist where non-technical people put all their stuff inside
without knowing what's happening under the hood and wonder that everything
explodes or becomes horribly slow at some point.
Andreas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 186 bytes
Desc: not available
Url : http://mail.zope.org/pipermail/zodb-dev/attachments/20060624/cdd9fab8/attachment-0001.bin
More information about the ZODB-Dev
mailing list