Mark Gibson wrote at 2006-3-22 19:28 -0700:
I'm struggling to weigh the cost of getObject() vs. the cost of adding more metadata to the catalog. I'll explain my situation.
I have 10,000 widgets cataloged. I do a path and date query that returns me maybe 12 of these. Then I have a choice of calling getObject().getData() on each of these, or I could add getData to the catalog metadata.
Does the cost of calling getObject() for a dozen objects justify creating a new metadata field?
More generally how does a large amount of metadata in the catalog affect performance of queries?
This is very difficult to say. We have used the standard metadata set for a long time. It contains "description". However, in our cases "description" was usually several kb large. We found that all catalog operations (lookup, indexing) were very slow and modifying catalog operations caused huge transaktions. The "IOBBuckets" containing the metadata records have been the culprit. Usually, they contain 45 metadata records. If you need one record, you in fact handle usually 45 of them. In our case, the buckets usually have been several 100 kB big -- much larger then usual container objects... On the other hand, if you have small metadata records (say, containing a few integers), then loading them may be much faster than loading all intermediate objects to your final object -- especially for applications such as determining statistics data (when you are processing a lot of objects). -- Dieter