[ZODB-Dev] Cache warm up time

[Roché Compaan]

Thanks, there are definitely some settings relating to the persistent
cache that I haven't tried before, simply because I've been avoiding
them.

I'd still be interested to know if one can leverage the Relstorage
memcache code for a ZEO cache, so if Shane doesn't get around to it
I'll have a stab at it myself. Loading objects from a persistent cache
will still cause IO so to me it seems that it would be a big win to
keep the cache in memory even while restarting.

--
Roché Compaan
Upfront Systems                   http://www.upfrontsystems.co.za




On Thu, Mar 7, 2013 at 9:35 PM, Leonardo Rochael Almeida
<leorochael at gmail.com> wrote:
> This mail from Jim at this list a couple of years ago was stocked full
> of nice tips:
>
> https://mail.zope.org/pipermail/zodb-dev/2011-May/014180.html
>
> In particular:
>
> - Yes, use persistent cache. Recent versions are reliable. Make it as
> large as resonable (e.g at most the size of your packed database, at
> least the size of objects that you want to be around after a restart).
>
> - Consider using zc.zlibstorage to compress the data that's stored in ZODB
>
> - set drop-cache-rather-verify to true on the client (avoid long
> restart time where your client is revalidating the ZEO cache)
>
> - set invalidation-age on the server to at least an hour or two so
>   that you deal with being disconnected from the storage server for a
>   reasonable period of time without having to verify.
>
> Cheers,
>
> Leo
>
> On Thu, Mar 7, 2013 at 3:54 PM, Roché Compaan
> <roche at upfrontsystems.co.za> wrote:
>> We have a setup that is running just fine when the caches are warm but
>> it takes several minutes after a restart before the cache warms up.
>> As per usual, big catalog indexes seem to be the problem.
>>
>> I was wondering about two things. Firstly, in 2011 in this thread
>> https://mail.zope.org/pipermail/zodb-dev/2011-October/014398.html
>> about zeo.memcache, Shane said that he could adapt the caching code in
>> RelStorage for ZEO. Shane do you still plan to do this? Do you think
>> an instance can restart without having to reload most objects into the
>> cache?
>>
>> Secondly, I was wondering to what extent using persistent caches can
>> improve cache warm up time and if persistent caches are usable or not,
>> given that at various times in the past, it was recommended that one
>> try and avoid them.
>>
>> --
>> Roché Compaan
>> Upfront Systems                   http://www.upfrontsystems.co.za
>> _______________________________________________
>> For more information about ZODB, see http://zodb.org/
>>
>> ZODB-Dev mailing list  -  ZODB-Dev at zope.org
>> https://mail.zope.org/mailman/listinfo/zodb-dev