At 07:12 PM 1/21/01 +0000, Steve Alexander wrote:
So, storing things in a Rack happens in a number of stages:
Your application interacts with the Rack The Rack (perhaps) stores the object persistently in its BTree The BTree is a collection of persistent ZODB objects The ZODB objects are stored as Python Pickles in a FileStorage
We can consider what the effect of storing 60 000 objects is at each of these interfaces.
The Rack shouldn't have a problem with 60 000 objects.
I doubt a BTree would have a problem.
The ZODB might not like accessing many large objects during a single transaction, as all those objects need to be in memory at once.
A FileStorage should have no problem reading 60 000 stored objects. However, if these objects are changing much, your Data.fs will grow fast. In any case, you may find undo and history screens take a long time to appear.
However, if you are using a Rack, you have a lot of choice about where you put your data. You can put frequently changed aspects of your data on the filesystem, and the rest in FileStorage for example.
Just to expand a little on the abov... Racks should scale at least as well, if not larger than a ZCatalog, given the same storage backing for the ZODB. This is because ZCatalog has to manage a minimum of one forward and reverse BTree for *each* index, plus another few BTrees for overall storage and housekeeping. Also, keyword and full text indexes store multiple BTree entries per object, so that's a factor as well. So don't worry about the Rack. If you're using a Rack, you can store the data anywhere, and you can index it in an RDBMS, LDAP directory, ZCatalog, or some combination thereof, using triggers to keep the data in sync.