[Zope] Large BTreeFolder2 batching/pagination

Ken Ara feedreader at yahoo.com
Mon Nov 2 05:48:46 EST 2009


I should also mention that one of these large collections contains objects with sequential, but not consecutive, numeric ids (timestrings) like some blogs. Users of this database can benefit from the ability to access items added around the same time.

With or without a catalog, how would one get the next or previous id in such a case?

--- On Mon, 11/2/09, Andreas Jung <lists at zopyx.com> wrote:

From: Andreas Jung <lists at zopyx.com>
Subject: Re: [Zope] Large BTreeFolder2 batching/pagination
To: "Ken Ara" <feedreader at yahoo.com>
Cc: zope at zope.org
Date: Monday, November 2, 2009, 11:36 AM

Am 02.11.09 11:28, schrieb Ken Ara:
> I agree it is hard to imagine, but I am just the web guy...
>
> If I dare to guess, I would say that what they want to do is improve
> access to the information for the users. The two million objects are
> strongly interlinked, so this is the navigation system used up to now.
> But why not open it up to browsing?
>

Because no human can deal in a reasonable way with 500k object..build
your *own*
custom and working navigation throughout the data records...presenting
the ZMI view for BTreefolders to a human is just sick.
>
> For the content team, it might also be nice to use the ZMI or similar
> when working with this content.
>
> We have not been successful to create a Catalog of this many objects.
> The process seemed to time out after many hours, probably
> hardware-bound. Also, the size of the resulting ZODB is of concern,
> but we may try again with the Catalog on a mounted database.
>
we have ZCatalogs with millions of objects..likely you are indexing
everything within one big transaction
without using savepoints or subtransactions..


>
-aj



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.zope.org/pipermail/zope/attachments/20091102/cbf27e60/attachment.html 


More information about the Zope mailing list