Too many ZODB object loads...
On the database Activity tab in the Control Panel, I am seeing a high number of Object Loads (400K per hour). My cache size if 60K from what I see in the Cache Parameters tab, each of the 4 threads is well within this limit. My database has about 1M objects. The symptom is that Zope appears slow when adding/editing content. Any idea if this is a problem? Suresh
Suresh V wrote at 2006-12-21 22:09 +0530:
On the database Activity tab in the Control Panel, I am seeing a high number of Object Loads (400K per hour). My cache size if 60K from what I see in the Cache Parameters tab, each of the 4 threads is well within this limit.
60 k is unusually large. Are you sure that this is not the combined size of all caches (such that each cache has a capacity of 15 k)? When the ZODB reported that it has read some number of objects, it is very likely that it did indeed.
... The symptom is that Zope appears slow when adding/editing content.
Do you use text indexing? with ranking information? The usual cosine (OKAPI) ranking requires a large amount of data. When you perform modifications, usually the problem is not the amount of objects read but the amount and size of objects written. Use "fsdump" to determine the size of your transactions. -- Dieter
Dieter Maurer wrote:
Suresh V wrote at 2006-12-21 22:09 +0530:
On the database Activity tab in the Control Panel, I am seeing a high number of Object Loads (400K per hour). My cache size if 60K from what I see in the Cache Parameters tab, each of the 4 threads is well within this limit.
60 k is unusually large. Are you sure that this is not the combined size of all caches (such that each cache has a capacity of 15 k)?
We hiked it up to 60K per thread because we have lots of memory and wanted to make Zope perform better. Is there a disadvantage to this?
When the ZODB reported that it has read some number of objects, it is very likely that it did indeed.
But even under very little site activity, it seems to be loading a lots of objects. Is this normal?
... The symptom is that Zope appears slow when adding/editing content.
Do you use text indexing? with ranking information?
The usual cosine (OKAPI) ranking requires a large amount of data.
Nope.
When you perform modifications, usually the problem is not the amount of objects read but the amount and size of objects written. Use "fsdump" to determine the size of your transactions.
I will check this. Thanks for your pointers. Suresh
Suresh V wrote at 2006-12-22 03:02 +0530:
... We hiked it up to 60K per thread because we have lots of memory and wanted to make Zope perform better. Is there a disadvantage to this?
Not, if you have enough RAM. Note that each connection has its own cache and that you control the cache size only by the the number of objects it may contain and not by the size of these objects. In a standard Zope setup, you cache capacity of 60 K means that up to 240 k objects can be cached. If you are using lots of "File" objects, then the "File" objects are split in 64 kB chunks. If your cache happens to be filled with such chunks, then you need about 15 GB just for ZODB caching purposes.
When the ZODB reported that it has read some number of objects, it is very likely that it did indeed.
But even under very little site activity, it seems to be loading a lots of objects. Is this normal?
It is quite easy to get the persistency design wrong. To analyse such bad design, I have instrumented our Zope copy to log information about the communication between a ZEO client (e.g. Zope) and the ZEO server). I attach the instrumented module. It is likely that you will not be able to replace the module in your Zope implementation but you can get a feeling how to approach such an instrumentation. Watch out for "_doLOG". -- Dieter
participants (2)
-
Dieter Maurer -
Suresh V