[ZODB-Dev] Speeding up ZODB (was "redis cache for RelStorage")

[Jim Fulton]

On Fri, May 6, 2011 at 4:21 PM, Shane Hathaway <shane at hathawaymix.org> wrote:
> On 05/06/2011 02:14 PM, Jim Fulton wrote:
>>>
>>> It sounds like you primarily need a bigger and faster cache.  If you
>>> want to make minimal changes to your setup, try increasing the size of
>>> your ZEO cache and store the ZEO cache on either a RAM disk (try mount
>>> -t tmpfs none /some/path) or a solid state disk.  Remember that seek
>>> time is 5-10 ms with spinning drives, so putting a ZEO cache on a
>>> spinning drive can actually kill performance.
>>
>> If this on Linux and you have enough RAM, the data should be in the
>> disk cache anyway, so I don't see any benefit to a RAM disk.
>
> If there is memory pressure then Linux will evict some of the cache from
> RAM, causing the ZEO cache to be much slower than the ZEO server.  I've seen
> that happen often.

If there is memory pressure and you take away ram for a ram disk, then you're
going to start swapping, which will give you other problems.

> "mount -t tmpfs" is an easy solution that has been available widely for a
> long time now.  You can even allow non-root users to do it by changing
> /etc/fstab.

I tried running ZODB tests off of a RAM disk created that way and got
lots of strange failures. :(

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton