[ZODB-Dev] Speeding up ZODB (was "redis cache for RelStorage")
Pedro Ferreira
jose.pedro.ferreira at cern.ch
Fri May 6 06:19:39 EDT 2011
Hello.
> Is your storage server CPU bound?
load average: 1.47, 1.34, 1.20
mpstat:
"""
11:40:13 AM CPU %user %nice %sys %iowait %irq %soft %steal
%idle intr/s
11:40:13 AM all 5.82 0.00 0.69 1.11 0.01 0.12 0.00
92.25 548.52
"""
I guess it's not very high, for an 8 core machine.
> Starting with ZODB 3.10, ZEO
> storage servers are multi-threaded. They have a thread for each
> client. We have a storage server that has run at 120% cpu on a 4-core
> box. Also, if you use zc.FileStorage, packing is mostly done in a
> separate process.
I didn't know this package even existed. I will give it a try, but in
any case our problems don't seem to be related to packing.
> A major source of slow down *can* be disk access times. How's IO wait
> on your server?
As you can see from the mpstat snapshot above, just around 1%. I have
checked iostat as well, and the number of transactions per second seems
to be very low considering the maximum allowed by the hardware.
> Look at how your application is using data. If you have requests that
> have to load a lot of data, maybe you can refactor your application to
> load fewer.
A quick check with nethogs shows values of network usage oscillating
between 100 and 200 KB/s. But I guess that if I were loading an
excessive amount of data, this value would be higher, no?
Thanks a lot for your answers.
Pedro
--
José Pedro Ferreira
Software Developer, Indico Project
http://indico-software.org
+-----------+
+ '``'--- `+ CERN - European Organization for Nuclear Research
+ |CERN| / + 1211 Geneve 23, Switzerland
+ ..__. \. + IT-UDS-AVC
+ \\___.\ + Office: 513-R-0042
+ / + Tel. +41227677159
+-----------+
More information about the ZODB-Dev
mailing list