[Zope] Thousands of Objects - how do they manifest? (newbie)

[Paul Winkler]

On Mon, Jan 06, 2003 at 03:04:39PM -0000, Crosbie Fitch wrote:
> 1) Are all objects in Zope persistent? ZClass instances? Python objects?

No, usually, and not necessarily.

Objects are persistent if their classes inherit (directly or
indirectly) from Persistent, and if the object is added to another
object which is an ObjectManager, e.g. a Folder instance.
Otherwise they are not persistent.

In order to meet these requirements, the object should be constructed
either as a ZClass or as a python Product. YOu can define and instantiate
classes in an External Method, but making them persistent would be Wrong.

The reason I said ZClass instances are "usually" persistent is that you
might add one to e.g. a TemporaryFolder whose sub-objects are
never written to disk.  But in most cases, you add instances to a Folder
and they are persistent.
 
> 2) Are all object instances visible as entries in a Zope UI folder? 

Yes.

> Is it
> possible to have persistent objects without them being visible? 

not with a standard Folder.

> Or does it
> not matter that the UI will attempt to list them all (millions or more)?

It would be very hard to use. :)
Have a look at Shane's BTreeFolder2, which is designed to
solve this kind of problem. 
http://hathaway.freezope.org/Software/BTreeFolder2

 
> 3) If I am expecting a very large number of objects would I always be better
> off using an external database? or are there circumstances where ZODB is
> quite able to look after them?

don't know, sorry. i've never pushed a zope folder beyond 100 or so items.
My biggest ZODB at the moment contains 150727 items for a total of
2.4 GB, according to the control panel.
 
> 4) If I need to be assured of being able to backup/restore the database
> and/or rebuild the site in the event of catastrophe does an external
> database provide any greater facility/reliability for achieving this?

depends on the database I guess.
You can back up zope just by copying the ZODB, but that gets interesting
when it's very large. I've heard on this list that copying a live ZODB might
be problematic if zope writes to it while you're backing it up. I don't
know if it leads to simply missing some updates in the copy, or 
worse corruption. So I'm stuck with some downtime while making the
backup copy, which is not insignificant with 2.4 GB of data.
One strategy I'm looking at to mitigate this would be:
1) restart zope in read-only mode.
2) make a copy of the zodb.
3) restart zope in normal read-write mode.

So I'd have only two very brief outages instead of one long one.

> Or is
> ZEO the thing to use?

ZEO does not address backup issues at all.
It just allows you to run multiple Zope servers from one ZODB.

-- 

Paul Winkler
http://www.slinkp.com
Look! Up in the sky! It's PERVERT  EPSILON!
(courtesy of isometric.spaceninja.com)