[Zodb-checkins] SVN: ZODB/branches/3.4/ More words;
update examples w/ new output.
Tim Peters
tim.one at comcast.net
Tue Jul 26 17:02:12 EDT 2005
Log message for revision 37435:
More words; update examples w/ new output.
Changed:
U ZODB/branches/3.4/doc/ZEO/trace.txt
U ZODB/branches/3.4/src/ZEO/simul.py
-=-
Modified: ZODB/branches/3.4/doc/ZEO/trace.txt
===================================================================
--- ZODB/branches/3.4/doc/ZEO/trace.txt 2005-07-26 20:29:49 UTC (rev 37434)
+++ ZODB/branches/3.4/doc/ZEO/trace.txt 2005-07-26 21:02:12 UTC (rev 37435)
@@ -46,11 +46,11 @@
can probably improve the cache performance (and hence your Zope application
server's performance) by increasing the ZEO cache size. This is normally
configured using key ``cache_size`` in the ``zeoclient`` section of your
-configuration file. The default cache size is 20 MB, which is very small.
+configuration file. The default cache size is 20 MB, which is small.
The stats.py tool shows its command line syntax when invoked without
arguments. The tracefile argument can be a gzipped file if it has a .gz
-extension. It will read from stdin (assuming uncompressed data) if the
+extension. It will be read from stdin (assuming uncompressed data) if the
tracefile argument is '-'.
Simulating Different Cache Sizes
@@ -66,37 +66,60 @@
Example, assuming the trace file is in /tmp/cachetrace.log::
- $ python simul.py -s 100 /tmp/cachetrace.log
- START TIME DURATION LOADS HITS INVALS WRITES FLIPS HITRATE
- Sep 4 11:59 38:01 59833 40473 257 20 2 67.6%
- $
+ $ python simul.py -s 4 /tmp/cachetrace.log
+ CircularCacheSimulation, cache size 4,194,304 bytes
+ START TIME DURATION LOADS HITS INVALS WRITES HITRATE EVICTS INUSE
+ Jul 22 22:22 39:09 3218856 1429329 24046 41517 44.4% 40776 99.8
-This shows that with a 100 MB cache size, the cache hit rate is
-67.6%. So let's try this again with a 200 MB cache size::
+This shows that with a 4 MB cache size, the cache hit rate is 44.4%, the
+percentage 1429329 (number of cache hits) is of 3218856 (number of load
+requests). The cache simulated 40776 evictions, to make room for new object
+states. At the end, 99.8% of the bytes reserved for the cache file were in
+use to hold object state (the remaining 0.2% consists of "holes", bytes freed
+by object eviction and not yet reused to hold another object's state).
- $ python simul.py -s 200 /tmp/cachetrace.log
- START TIME DURATION LOADS HITS INVALS WRITES FLIPS HITRATE
- Sep 4 11:59 38:01 59833 40921 258 20 1 68.4%
- $
+Let's try this again with an 8 MB cache::
-This showed hardly any improvement. So let's try a 300 MB cache
-size::
+ $ python simul.py -s 8 /tmp/cachetrace.log
+ CircularCacheSimulation, cache size 8,388,608 bytes
+ START TIME DURATION LOADS HITS INVALS WRITES HITRATE EVICTS INUSE
+ Jul 22 22:22 39:09 3218856 2182722 31315 41517 67.8% 40016 100.0
- $ python2.0 simul.py -s 300 /tmp/cachetrace.log
- ZEOCacheSimulation, cache size 300,000,000 bytes
- START TIME DURATION LOADS HITS INVALS WRITES FLIPS HITRATE
- Sep 4 11:59 38:01 59833 40921 258 20 0 68.4%
- $
+That's a huge improvement in hit rate, which isn't surprising since these are
+very small cache sizes. The default cache size is 20 MB, which is still on
+the small side::
-This shows that for this particular trace file, the maximum attainable
-hit rate is 68.4%. This is probably caused by the fact that nearly a
-third of the objects mentioned in the trace were loaded only once --
-the cache only helps if an object is loaded more than once.
+ $ python simul.py /tmp/cachetrace.log
+ CircularCacheSimulation, cache size 20,971,520 bytes
+ START TIME DURATION LOADS HITS INVALS WRITES HITRATE EVICTS INUSE
+ Jul 22 22:22 39:09 3218856 2982589 37922 41517 92.7% 37761 99.9
-The simul.py tool also supports simulating different cache
-strategies. Since none of these are implemented, these are not
-further documented here.
+Again a very nice improvement in hit rate, and there's not a lot of room left
+for improvement. Let's try 100 MB::
+ $ python simul.py -s 100 /tmp/cachetrace.log
+ CircularCacheSimulation, cache size 104,857,600 bytes
+ START TIME DURATION LOADS HITS INVALS WRITES HITRATE EVICTS INUSE
+ Jul 22 22:22 39:09 3218856 3218741 39572 41517 100.0% 22778 100.0
+
+It's very unusual to see a hit rate so high. The application here frequently
+modified a very large BTree, so given enough cache space to hold the entire
+BTree it rarely needed to ask the ZEO server for data: this application
+reused the same objects over and over.
+
+More typical is that a substantial number of objects will be referenced only
+once. Whenever an object turns out to be loaded only once, it's a pure loss
+for the cache: the first (and only) load is a cache miss; storing the object
+evicts other objects, possibly causing more cache misses; and the object is
+never loaded again. If, for example, a third of the objects are loaded only
+once, it's quite possible for the theoretical maximum hit rate to be 67%, no
+matter how large the cache.
+
+The simul.py script also contains code to simulate different cache
+strategies. Since none of these are implemented, and only the default cache
+strategy's code has been updated to be aware of MVCC, these are not further
+documented here.
+
Simulation Limitations
----------------------
@@ -118,4 +141,4 @@
requests for O will continue to be simulated cache misses, although in a
real cache they'll likely be cache hits. On the other hand, the
simulated cache doesn't need to evict any objects to make room for O, so it
- may enjoy further cache hits on objects a real cache would need to evict.
+ may enjoy further cache hits on objects a real cache would have evicted.
Modified: ZODB/branches/3.4/src/ZEO/simul.py
===================================================================
--- ZODB/branches/3.4/src/ZEO/simul.py 2005-07-26 20:29:49 UTC (rev 37434)
+++ ZODB/branches/3.4/src/ZEO/simul.py 2005-07-26 21:02:12 UTC (rev 37435)
@@ -299,7 +299,7 @@
from ZEO.cache import ZEC3_HEADER_SIZE
class CircularCacheSimulation(Simulation):
- """Simulate the ZEO 3.0a cache."""
+ """Simulate the ZEO 3.0 cache."""
# The cache is managed as a single file with a pointer that
# goes around the file, circularly, forever. New objects
@@ -572,7 +572,7 @@
# CAUTION: It's most likely that none of the simulators below this
# point work anymore. A great many changes were needed to teach
# CircularCacheSimulation (above) about MVCC, including method signature
-# changes and changes in cache file format, and none of the others simulator
+# changes and changes in cache file format, and none of the other simulator
# classes were changed.
#############################################################################
More information about the Zodb-checkins
mailing list