Populating a "huge" ZODB
Hi everybody, How would you populate a "huge" ZODB with hundreds of thousands of objects which have no specific reason to be hierarchically distributed in a complex tree of folders and sub-folders? They could in fact all be in the same folder. It seems to be OK for hundreds of objects, but afterwards there is an incremental cost : each time you add a new object, there is a check on the existence of an identical ID before adding/creating it. When you reach the level of already existing thousands/tens of thousands/hundreds of thousands objects in the same folder it seems you are in serious trouble... I know the question is a bit naive, and that usually you have a data structure which dictates a naturally "hierarchy" of folders which means that no folder would contain maybe more than a few hundred objects or sub-folders. Maybe I should "artificially" design an ID structure which would correspond to a hierarchy : for instance ojects would be identified by numbers with 7 digits, each digit corresponding to a lower level in the hierarchy? It means that an object with ID 2343789 would be placed in the "/2/3/4/3/7/8/9" sub-folder. What do you think? Any other idea? Pierre Godefroy 52 rue des Archives 75004 PARIS FRANCE Tél. : +33 (0)1 42 74 46 05
On Fre, 2002-06-21 at 11:57, Pierre Godefroy wrote: [ huge folder ] Have a look at BTreeFolder, the whole structure is stored in a btree, so a lookup for an id should be pretty fast... greetings, Igor -- ScanPlus GmbH NOC Ulm - Germany - Griesbadgasse 7-13 - D 89073 Ulm TEL +49 731 920 13 100 - FAX +49 731 920 13 290 eMail: support@scan-plus.de Amtsgericht Ulm - HRB3220 - Geschaeftsfuehrer: Juergen Hoermann
participants (2)
-
Igor Stroh -
Pierre Godefroy