From experience I find that BTrees are very fast to write to and pick out items from. Even in the millions. (Never gone into the tens of millions or further) Also, when it comes to browsing stuff I find SQL faster and easier to work with. An added advantage of a RDBMS is that you get the indexing seamlessly built in (no need to bridge zbrain.getObject()) and it makes it easier to optimize and figure out which indexes help and which indexes slow you down which is something that is far from obvious with a ZCatalog approach.
2009/4/25 Morten W. Petersen <morten@nidelven-it.no>:
Hi,
I'm considering building a large scale, but small in features site. It will contain lots of small objects (millions, tens of millions, hundreds of millions) of objects, where each object has a couple of strings and maybe some other light attributes.
So far, I've been contemplating disabling undo (if that's possible), and using BTree structures, maybe segmenting objects into different groups (folders) to further speed up lookups. Scalability is also an issue, should I consider using RelStorage? Should I consider using the ZCatalog for faster lookups?
Has anyone else developed something similar? Are there Zope product examples out there that fit the bill?
-Morten
-- Morten W. Petersen Manager Nidelven IT Ltd
Phone: +47 45 44 00 69 Email: morten@nidelven-it.no
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
-- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com