[Zope-dev] any limits on object number?

[Nagarjuna G.]

On Wed, 2004-07-14 at 14:20, Andreas Jung wrote:
> --On Mittwoch, 14. Juli 2004 13:46 Uhr +0530 "Nagarjuna G." 
> <nagarjun at gnowledge.org> wrote:
> 
> > We are builiding a large portal using Zope.  We need to create a large
> > number of objects.
> 
> How large?

in one case, we want the metadata of all the wikipedia contents, that
will be around 300,000 objects.  

>   The data component of the objects is small, but each
> > object carries lots of metadata.  My question is: Is there any limit on
> > the number of objects in a given folder?
> 
> No, but you should use BTreeFolder2 in any case and don't use the standard
> Zope folder!

thanks, this is certainly going to help, I will try inheriting this
class. 

> 
> > I am not planning to use any
> > external RDBMS. Are there any known performace isssues when the numbeer
> > of objects increase, particularly when we store them in the same folder?
> >
> >
> 
> Maybe tell us some details.....

The project is GNOWSYS, an application that builds semantic web portals
on top of Zope.   

When we tried to upload objects from wikipedia's sqldump of after about
40,000 objects in a folder the script went for a toss. roughly each time
we ran the script the same problem.  even to visit the folder of even
1000 objects through ZMI is a pain.  I think that is where BTreeFolder2
will be very useful.

I am also thinking to use a squid cache style of storing objects in
zodb, where the objects go into many subfolders (i dont know what is
this method called technically) with  several sub folders with names
alphabetized/numbered.  this way i can store large number within zodb
though not in the same folder/BTreefolder2.

Should I use both methods? 

Not related to Zope, but what about filesystems? there must be a limit
on the number of files within one folder depending on which filesystem
one uses.   

Thanks
Nagarjuna