[ZODB-Dev] Runaway cache size

[AFoglia at princeton.com]

Gary Poster <gary at zope.com> wrote on 07/31/2008 01:07:39 PM:

> I'm the primary zc.catalog author.
> 
> He is running read-only, so savepoints shouldn't be very pertinent.
> 
> The original poster is correct that the cache is cleaned out at 
> transaction boundaries (or savepoints, as Alan said).
> 
> I don't have experience with running the db in readonly mode in 
> production.  That said, if you have obvious boundaries for your 
> searches, I suspect that beginning a transaction at the boundary would 
> probably do the trick, and also work (with invalidations and the like) 
> if you ever have a readonly connection to a ZEO database server that 
> is getting updates from another source.
> 
> While it's interesting to contemplate the knob that Alan mentioned, 
> I'm reasonably confident that this logic belongs in the app, not the 
> index or catalog.

        I was thinking it this logic would be in the ZODB.DB since it (or 
something else in ZODB) is already responsible for clearing out the cache 
at transaction boundaries, and it should be aware the FileStorage is 
read_only.  But it doesn't sound like it is.  Also, it sounds like the 
cache is not for keeping any objects in memory to avoid rereading from 
disk (excepting those changed that need to be written in the next 
transaction), but only to keep some objects in memory across transaction 
boundaries.

        For now I've put it into the "app" (really, a thrown-together 
script) to call cacheMinimize on the DB object after dealing with the 
results of each search.  If we were going to stick with ZODB, I'd work on 
handling it in my own code, but we're moving away from ZODB (for 
managerial reasons).

-- 
Anthony Foglia
Princeton Consultants
(609) 987-8787 x233
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.zope.org/pipermail/zodb-dev/attachments/20080731/a153890d/attachment.html