[ZODB-Dev] Cache warm up time

[Claudiu Saftoiu]

I'd be curious to know what your results are, whichever path you decide to
take! Might help inform me as to what might help on my server...

One thing I haven't yet understood is - how come the ZEO server itself
doesn't have a cache? It seems that would be a logical place to put one as
the ZEO server generally rarely gets restarted, at least for the use case
of running both the ZEO server and the clients on the same machine.

On Fri, Mar 8, 2013 at 1:46 AM, Roché Compaan <roche at upfrontsystems.co.za>wrote:

> Thanks, there are definitely some settings relating to the persistent
> cache that I haven't tried before, simply because I've been avoiding
> them.
>
> I'd still be interested to know if one can leverage the Relstorage
> memcache code for a ZEO cache, so if Shane doesn't get around to it
> I'll have a stab at it myself. Loading objects from a persistent cache
> will still cause IO so to me it seems that it would be a big win to
> keep the cache in memory even while restarting.
>
> --
> Roché Compaan
> Upfront Systems                   http://www.upfrontsystems.co.za
>
>
>
>
> On Thu, Mar 7, 2013 at 9:35 PM, Leonardo Rochael Almeida
> <leorochael at gmail.com> wrote:
> > This mail from Jim at this list a couple of years ago was stocked full
> > of nice tips:
> >
> > https://mail.zope.org/pipermail/zodb-dev/2011-May/014180.html
> >
> > In particular:
> >
> > - Yes, use persistent cache. Recent versions are reliable. Make it as
> > large as resonable (e.g at most the size of your packed database, at
> > least the size of objects that you want to be around after a restart).
> >
> > - Consider using zc.zlibstorage to compress the data that's stored in
> ZODB
> >
> > - set drop-cache-rather-verify to true on the client (avoid long
> > restart time where your client is revalidating the ZEO cache)
> >
> > - set invalidation-age on the server to at least an hour or two so
> >   that you deal with being disconnected from the storage server for a
> >   reasonable period of time without having to verify.
> >
> > Cheers,
> >
> > Leo
> >
> > On Thu, Mar 7, 2013 at 3:54 PM, Roché Compaan
> > <roche at upfrontsystems.co.za> wrote:
> >> We have a setup that is running just fine when the caches are warm but
> >> it takes several minutes after a restart before the cache warms up.
> >> As per usual, big catalog indexes seem to be the problem.
> >>
> >> I was wondering about two things. Firstly, in 2011 in this thread
> >> https://mail.zope.org/pipermail/zodb-dev/2011-October/014398.html
> >> about zeo.memcache, Shane said that he could adapt the caching code in
> >> RelStorage for ZEO. Shane do you still plan to do this? Do you think
> >> an instance can restart without having to reload most objects into the
> >> cache?
> >>
> >> Secondly, I was wondering to what extent using persistent caches can
> >> improve cache warm up time and if persistent caches are usable or not,
> >> given that at various times in the past, it was recommended that one
> >> try and avoid them.
> >>
> >> --
> >> Roché Compaan
> >> Upfront Systems                   http://www.upfrontsystems.co.za
> >> _______________________________________________
> >> For more information about ZODB, see http://zodb.org/
> >>
> >> ZODB-Dev mailing list  -  ZODB-Dev at zope.org
> >> https://mail.zope.org/mailman/listinfo/zodb-dev
> _______________________________________________
> For more information about ZODB, see http://zodb.org/
>
> ZODB-Dev mailing list  -  ZODB-Dev at zope.org
> https://mail.zope.org/mailman/listinfo/zodb-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.zope.org/pipermail/zodb-dev/attachments/20130308/d8defde6/attachment.html>