[Zope] Populating a "huge" ZODB

[Pierre Godefroy]

Hi everybody,

How would you populate a "huge" ZODB with hundreds of thousands of objects=
=20
which have no specific reason to be hierarchically distributed in a complex=
=20
tree of folders and sub-folders?

They could in fact all be in the same folder. It seems to be OK for=20
hundreds of objects, but afterwards there is an incremental cost : each=20
time you add a new object, there is a check on the existence of an=20
identical ID before adding/creating it. When you reach the level of already=
=20
existing thousands/tens of thousands/hundreds of thousands objects in the=20
same folder it seems you are in serious trouble...

I know the question is a bit naive, and that usually you have a data=20
structure which dictates a naturally "hierarchy" of folders which means=20
that no folder would contain maybe more than a few hundred objects or=20
sub-folders.

Maybe I should "artificially" design an ID structure which would correspond=
=20
to a hierarchy : for instance ojects would be identified by numbers with 7=
=20
digits, each digit corresponding to a lower level in the hierarchy? It=20
means that an object with ID 2343789 would be placed in=20
the  "/2/3/4/3/7/8/9" sub-folder.

What do you think? Any other idea?


Pierre Godefroy
52 rue des Archives
75004 PARIS
FRANCE

T=E9l. : +33 (0)1 42 74 46 05