[Zope] BTreeFolder for "huge" ZODB (was : populating a "huge" ZODB)

Fri, 21 Jun 2002 13:33:19 +0200

Hi Pierre,

I would suggest to store your bibliographic records on a external DB =
like MySQL, PostgreSQL or Oracle.
Or if you have a lot of unstructured data use a XML-Database like =
Xindice.

You would than create an application on zope to work with these =
data-records.

Regards,
Dirk

----- Original Message -----=20
From: "Pierre Godefroy" <pierre.godefroy@noos.fr>
To: "Oliver Marx" <oliver@tekk.dk>; <zope@zope.org>
Sent: Friday, June 21, 2002 12:55 PM
Subject: RE: [Zope] BTreeFolder for "huge" ZODB (was : populating a =
"huge" ZODB)

Oliver,

Yes, in fact it would be for the management of bibliographic records.

Pierre

At 12:43 21/06/2002 +0200, Oliver Marx wrote:
>If I understand you right then you want to use ZODB as th=E9 DB for an
>application?
>
>\Oliver
>
> >>-----Original Message-----
> >>From: zope-admin@zope.org [mailto:zope-admin@zope.org]On Behalf Of
> >>Pierre Godefroy
> >>Sent: 21. juni 2002 12:37
> >>To: zope@zope.org
> >>Subject: [Zope] BTreeFolder for "huge" ZODB (was : populating a =
"huge"
> >>ZODB)
> >>
> >>
> >>Thank you very much Igor, however I am still a bit afraid because I =
see a
> >>scrolling window for the products contrained by the BTreeFolder on =
the
> >>product homepage
> >>(http://www.zope.org/Members/hathawsh/BTreeFolder). In the
> >>exmple given (4096 items), it could still be OK, but I think having
> >>1,000,000 + objects might be a problem. There is an interface
> >>problem here
> >>: the contents of such a folder can only be reasonably displayed in
> >>successive batches. (in fact it is also another aspect of the =
problem of
> >>having thousands of objects in an ordinary folder : if you click on =
the
> >>folder to open it, the Management interface will hang)
> >>
> >>Pierre
> >>
> >>
> >>>On Fre, 2002-06-21 at 11:57, Pierre Godefroy wrote:
> >>>[ huge folder ]
> >>>Have a look at BTreeFolder, the whole structure is stored in a =
btree, so
> >>>a lookup for an id should be pretty fast...
> >>>greetings,
> >>>Igor
> >>
> >>--
> >>>Hi everybody,
> >>>
> >>>How would you populate a "huge" ZODB with hundreds of thousands
> >>of objects
> >>>which have no specific reason to be hierarchically distributed in a
> >>>complex tree of folders and sub-folders?
> >>>
> >>>They could in fact all be in the same folder. It seems to be OK for
> >>>hundreds of objects, but afterwards there is an incremental cost : =
each
> >>>time you add a new object, there is a check on the existence of an
> >>>identical ID before adding/creating it. When you reach the level of
> >>>already existing thousands/tens of thousands/hundreds of
> >>thousands objects
> >>>in the same folder it seems you are in serious trouble...
> >>>
> >>>I know the question is a bit naive, and that usually you have a =
data
> >>>structure which dictates a naturally "hierarchy" of folders which =
means
> >>>that no folder would contain maybe more than a few hundred objects =
or
> >>>sub-folders.
> >>>
> >>>Maybe I should "artificially" design an ID structure which would
> >>>correspond to a hierarchy : for instance ojects would be identified =
by
> >>>numbers with 7 digits, each digit corresponding to a lower level in =
the
> >>>hierarchy? It means that an object with ID 2343789 would be placed =
in
> >>>the  "/2/3/4/3/7/8/9" sub-folder.
> >>>
> >>>What do you think? Any other idea?
> >>
> >>
> >>
> >>
> >>
> >>Pierre Godefroy
> >>52 rue des Archives
> >>75004 PARIS
> >>FRANCE
> >>
> >>T=E9l. : +33 (0)1 42 74 46 05
> >>
> >>
> >>
> >>_______________________________________________
> >>Zope maillist  -  Zope@zope.org
> >>http://lists.zope.org/mailman/listinfo/zope
> >>**   No cross posts or HTML encoding!  **
> >>(Related lists -
> >> http://lists.zope.org/mailman/listinfo/zope-announce
> >> http://lists.zope.org/mailman/listinfo/zope-dev )
> >>---
> >>Incoming mail is certified Virus Free.
> >>Checked by AVG anti-virus system (http://www.grisoft.com).
> >>Version: 6.0.371 / Virus Database: 206 - Release Date: 13-06-2002
> >>
>---
>Outgoing mail is certified Virus Free.
>Checked by AVG anti-virus system (http://www.grisoft.com).
>Version: 6.0.371 / Virus Database: 206 - Release Date: 13-06-2002

Pierre Godefroy
52 rue des Archives
75004 PARIS
FRANCE

T=E9l. : +33 (0)1 42 74 46 05

_______________________________________________
Zope maillist  -  Zope@zope.org
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -=20
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )