[Zope] ZODB vs Relational Databases

[sean.upton@uniontrib.com]

Applications that scale beyond thousands of objects in a single container
can perform quite well, but only if you code your app right; you need to
have the application take into account using the Catalog as much as possible
to avoid excessive penalty for traversing to too many objects or relying too
much upon ObjectManager (folder) interfaces for accessing multiple objects.
Traversing to an object is expensive, if you add up many objects, largely
because of security machinery and other concerns.  Using cached data in the
Catalog can help you avoid this performance penalty, even for applications
that do nothing more than list items in a given folder in a sorted and/or
batched manner.

Tips:
- Download and use the BTreeFolder2 product for any folder with many objects
- Subclass BTreeFolder2, and back it with a ZCatalog, and associated UI
	- Assumes working knowledge of writing Zope Python Products
- Use the catalog wherever you can, and only load the objects you need to
access
- Avoid ObjectManager methods like objectItems(), objectValues(),
objectIds() on large containers, use a catalog to get the same results,
which can of-course be filtered, sorted, and batched!

Good Luck,
Sean

-----Original Message-----
From: Michael Long [mailto:mlong@datalong.com]
Sent: Monday, March 10, 2003 7:19 AM
To: Jeremy Cowgar; zope@zope.org
Subject: Re: [Zope] ZODB vs Relational Databases


It is encouraging to hear that the zope can handle some fairly large
numbers. This was one of my biggest concerns. How is performance when
retieving this data? Have you seen any degradation as the number of has
grown, specifically have you seen any threshold that should be avoided?

Thanks,
Mike
> 
> I asked the same question not too long ago and the answer I got was to
give it 
> a try and see what it will handle. So far I have been impressed. It
requires 
> re-thinking if you are a RDBMS guy like me (and like you sound). For 
> instance, do not store relations by id, or key, or other values. It'll
slow 
> things down, store them by path.
> 
> Say for instance you have a class and students. Each student can
belong to a 
> class....
> 
> /class/math
> /class/history
> 
> Those are the paths to the classes. Now in your student table, store that 
> path. That is the relation.
> 
> Hope this helps. I have loaded it with 31102 * 5 objects. (5 Bible 
> translations at 31102 verses (objects) per).
> 
> If you do decide to use RDBMS however, it plugs into zope very nicely,
so no 
> concerns their either.
> 
> Jeremy
> 
> _______________________________________________
> Zope maillist  -  Zope@zope.org
> http://mail.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists - 
>  http://mail.zope.org/mailman/listinfo/zope-announce
>  http://mail.zope.org/mailman/listinfo/zope-dev )
> 
> 



_______________________________________________
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )