[Zope-DB] Maximum Zope objects in a database

[Jim Penny]

On Tue, Jul 30, 2002 at 12:01:01AM +0200, Le grande pinguin wrote:
> On Mon, Jul 29, 2002 at 05:24:25PM -0400, Jim Penny wrote:
> > 
> > If you are using an external database, this is really not a zope
> > question at all.  It comes down to how fast the database can prepare and
> > execute the query.  And this depends on all sorts of database
> > optimization, on system RAM, probably filesystem layout, etc.
> 
> Correct me if i'm wrong, but once you access these objects they move
> from the database to core memory. Depending on the access pattern for
> these objects you might run into problems with python's garbage collector
> as well. Even accessing only a few objects at a time but changeing these
> few will create excessive creation of objects that need to be collected
> soon afterwards. 

Yes, you may be right, but that hardly matters.

You will not successfully display 200,000 lines in a table in any
browser.  I am assuming that the original user has sane requirements,
i.e., a lot of data which would be collated and looked at in reasonably
small batches.  And that the queries are being launched by humans, and 
not by benchmarking suites.

It is true that for each response, zope prepares a so-called results
dictionary, and that the results dictionary is a pretty heavy-weight
object.

But, in ordinary practice, this does not matter.  Building the results
structure, as heavy as it is, is still easier than the database query's
parsing, optimization and fetch.

And, if you are returning so many results that the browser cannot
display them, you need to seriously rethink your query strategy.

> 
> > Note that the ZSQL methods typically simply send a query as a string to
> > the database system.  They do not generally do prepare operations,
> > stored queries, etc.
> 
> But you can call stored procedures.

You may be right.  I was mis-remembering that the Python Database API
Specification v2.0, http://www.python.org/peps/pep-0249.html, provided
no support for stored procedures.  It does allow them.

However, I am not sure that any DA's allow them, with the exception of
the Oracle DA, and maybe the Sybase DA.

Do not count on stored procedures being available.  It is dependent both
on the database system and the DA.

Jim
> 
> 
> Ralf Mattes
> 
> > This sounds like the kind of situation where you do want to use an
> > external database.  I have nothing nearly that large.  The largest I
> > have is about 64,000 records.  But, I see no zope-related bottleneck.
> > 
> > Jim Penny
> > > 
> > > 
> > > _______________________________________________
> > > Zope-DB mailing list
> > > Zope-DB@zope.org
> > > http://lists.zope.org/mailman/listinfo/zope-db
> > > 
> > 
> > _______________________________________________
> > Zope-DB mailing list
> > Zope-DB@zope.org
> > http://lists.zope.org/mailman/listinfo/zope-db
>