[Zope] zeo cache-size tuning - number of obj. vs MB

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Gaute Amundsen wrote:
> Hi
> 
> What's the difference between the to cache-size settings here?
> I've not been able to find a good description to guide me in finding the
> right values.
> 
> <zodb_db main>
>   mount-point /
>   cache-size 10000          1.
>   <zeoclient>
>     server foo:9080
>     storage main   
>     name zeostorage  
>     var $INSTANCE/var
>     cache-size 20MB          2.
>   </zeoclient>
> </zodb_db>
> 
> As far as I've been able to determine, the second one 2. refers to a
> separate cache at the zeo connection level.

The ZEO client cache is a disk-based cache, separate from the RAM-based
cache used by any ZODB connection.  It doesn't use RAM, per se, except
for an in-memory index of OID -> (layer, offset).

> Running 9 zeo clients (2.7.9) like this on a frontend with 4G ram,
> varnish, apache, and haproxy. Tethering on the edge of too much
> swapping. Needing to cut down..

Assuming you are running with the default number of "worker threads"
(4):  10000 objects in the cache * 4 active connections per appserver *
9 appservers means you are caching 360,000 objects in RAM.  If they are
putting you into swap, then you are averaging something like 10Kb per
object, which is pretty big.

> Would it be better to spend all the memory on 1?
> Is there a good way to find a balance?

I think I might cut down on the number of appservers on that host, if it
truly is hitting swap.  Or trim the number of threads per appserver down
from 4 to 2 or 3.



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAktEUIQACgkQ+gerLs4ltQ6pYwCfX4g/ZVfAnyJNQ4Y1njnSx1HV
6QEAnAg/UHsCEQSx6rUpj5K9X2JkoGdQ
=7ya0
-----END PGP SIGNATURE-----