[ZODB-Dev] Location of transient caches

[Jim Fulton]

On Fri, Feb 19, 2010 at 10:26 AM, Andy Theyers <andy.theyers at isotoma.com> wrote:
> Hi all
>
> We have a Debian server running 3 Zope instances, each mounting its
> database from a single ZEO instance on the same server.
>
> Until very recently we had been running each Zope with a persistent
> cache (using the 'client' directive in the zeo_client stanza of
> zope.conf).  Each cache was 300MB, as recommended by various buildout
> scripts and other "how to" documents I had read.

Hm, that's interesting. I hadn't seen that recommendation.
How big is your database?  How are you talking to the ZEO server?
TCP or unix domain sockets. Given that they're on the same machine,
I'd recommend the later, in which case, you may not get that much benefit
from the caches. You might try using smaller caches.

> While under this configuration the site failed due to cache corruption.
>  This caused us to remove the persistent cache option, again as
> recommended in various mailing list posts.

Note that the latest versions of ZODB (3.8 and 3.9) have fixed
most or all (not quite sure :) of the cache corruption issues.

> However, as we are using the default Debian disk partitioning (whereby /
> is only 1GB) we ran out of disk space, as the transient caches appear to
> be being created in /tmp
>
> Is there any way I can change the location of the transient cache files,
> putting them on a partition with more space?  I've been scouring the
> source but am yet to come up with the bit that creates the temporary
> cache files.  It's not an area of Zope that I've got any experience of.
>
> Any help would be greatly appreciated.

ZEO uses the tempfile module for transient caches, which
honors several environment variables to control where
temporary files go. You can also control this by setting a tempfile
module attribute from Python. See the tempfile documentation.

(I'd be happy to accept a patch/checkin with test to make this
 configurable from the ZEO configuration.)

Jim

-- 
Jim Fulton