Hello, In light of ZEO becoming open source, I'm still extremely excited, I want to revisit some questions I threw out in a previous thread about Zope's scalability. Can I architect a website like this using different Zope Storage Servers for the various directories or 'would be' tables in an RDBMS? Yes, I understand the impact of the OS filesystem concerning ZODB size. Say for example I have a large database. Like this: ZAuto ZAuto/parts/ ZAuto/manufacturers/ ZAuto/distributors/ ZAuto/distributors/...(orders, address, preferences, etc.) ZAuto/resellers/...(orders, address, preferences, etc.) ... Let's say that ZAuto/parts is a 5+gb directory which is reasonably stable updated monthly. ZAuto/manufacturers isn't as large but is also stable with periodic updates. ZAuto/distributors and ZAuto/resellers are much more dynamic with changes and additions. ZAuto/parts could be optimized for search performance. It only needs to be backed up periodically when bulk changes are made. It will relatively almost never need packed. It could sit on it's own drive, own machine, etc. It would be nice to be able to partition the database into stable/volatile files. Since the stable partition is also the largest, it seems this would also help performance of the more dynamic section. Also, when comments are made about Zope's ZODB or ZSS not being appropriate for heavy write apps, does this refer to apps doing "updates" on objects or doing "inserts" or both using SQL terminology? Would structuring an app like above help in write situations? I am trying to determine the desirability or necessity of an RDBMS for my idea. I like keeping it in Zope if reasonable and ZEO can definitely help. If need be I'll go to an RDBMS for certain data. Thanks for any help or clues. Jimmie Houchin
Jimmie Houchin wrote:
Hello,
Can I architect a website like this using different Zope Storage Servers for the various directories or 'would be' tables in an RDBMS? Yes, I understand the impact of the OS filesystem concerning ZODB size.
ZEO is transparent to the application, it is a _singular storage_ meaning that all the issues of multiple databases still apply, re, they are not yet supported.
It would be nice to be able to partition the database into stable/volatile files. Since the stable partition is also the largest, it seems this would also help performance of the more dynamic section.
What this needs is the ZDBF (Zope DataBase Framework). This is a layer that sits above ZODB. Currently, ZODB takes care of all these issues (or punts on them) when really a higher level abstraction should handle multiple database issues (including any future extensions to the Relational/Object model). Obviously, this is a big task. See paul's email about resources. ;)
Also, when comments are made about Zope's ZODB or ZSS not being appropriate for heavy write apps, does this refer to apps doing "updates" on objects or doing "inserts" or both using SQL terminology?
First, keep in mind that to a certain degree, all databases are sensitive to the high-write issue. ZODB is more sensitive because it's conflict resolution is optimisitic. FileStorage is also sensitive because of the inherent behavior of filesystems, and ZEO is sensitive because for every object written, an invalidation message must be sent to all other ZEOs so that they are given the chance to invalidate the object in their cache. This also increases chances of collisions because the Time To Write a record is extended for varous reasons (network delay, invalidation delay, etc...).
Would structuring an app like above help in write situations?
Not without ZDBF. Multiple database support would allow you to use different, less write sensitive storages like a RelationalStorage or a BerkeleyStorage.
I am trying to determine the desirability or necessity of an RDBMS for my idea. I like keeping it in Zope if reasonable and ZEO can definitely help. If need be I'll go to an RDBMS for certain data.
Do some benchmarks, ZODB is very fast if you are a little more clever about how you change your objects. If you need more write intensiveness go relational. -Michel
participants (2)
-
Jimmie Houchin -
Michel Pelletier