[Zope] Caching on-the-fly resized photos (implemented)

[Ron Bickers]

> -----Original Message-----
> From: zope-admin@zope.org [mailto:zope-admin@zope.org]On Behalf Of
> Robert Rottermann

> I have no displays created automatically. However the user can
> define a size
> of the picture she wants to use. From this I then create a display.
> Since the user will check here "story" while working on it the
> displays are
> created at a very early state in the game where waiting is tolerable.
>
> The idea of timing out displays is appealing.

Ok.  I've implemented something that seems like a good idea, but I'm not
sure how it will fly in production.  Also, I don't think it directly
addresses the above scenario.  I could probably extend what I have done to
create displays (not just the images of predefined displays, but actual new
display sizes) when a custom size is requested.  I could then tag it such
that when it times out the entire display (not just the stored image) is
removed.  Robert, would that cover what you're looking for?  Read on and it
should make more sense...

The current CVS at sf.net/projects/zopephoto has options for on-the-fly
generation of predefined display sizes and a timeout (in minutes) associated
with each Photo.  Each of these settings can be set in a Photo Folder to
control the settings in all contained Photos.

If the "pregenerate" option is chosen and timeout is set to zero, Photos
behave exactly has they have to date;  all displays are generated when the
Photo is added/updated and the are never purged.  If the pregenerate option
is not chosen, displays will be generated on-the-fly as they are requested.

The timeout is the tricky part.  Ideally, every time a display is requested,
an *access* time of that object would be updated and thus the age of the
display would be reset to 0.  This way, commonly accessed display sizes
would never be purged.  Since ZODB doesn't keep an access time, I could just
use the mod time and update it when the display is requested.  The problem
with this is that it would cause a large number of writes to the ZODB, and
since ZODB is append only, it would quickly bloat.

Since I couldn't come up with a better way to track access time, I
compromised.  Every time a display is requested, the age is compared with
the timeout.  If the age is more than half the timeout, the display's mod
time is reset to zero and any other display sizes whose age is greater than
timeout are purged.  With regular ZODB packs (which I assume most people do
anyway), and high timeouts (like several days), this might actually work
okay.

In addition, you can run manage_purgeDisplays() or manage_cleanDisplays()
which purges all displays (regardless of age) and purges all expired
displays, respectively.  Running these on the Photo Folder will process all
contained Photos.  There is a button in the management interface for each of
these.

I can see there might be the desire to have certain displays never expire
(such as a thumbnail), but instead of adding extra features for that, I'll
assume for now that in practice, thumbnails would be viewed often enough to
not be purged anyway.  We'll see.

Also, manage_purgeDisplays() and manage_cleanDisplays() each take an
'exclude' parameter that is a tuple of displays to *not* purge/clean.  In
each case, excluded displays will have their age reset to zero.  From the
management interface, if you "check" a display(s) and then purge/clean,
those displays will be excluded.  This might seem backwards, but I think
it's more practical.

If you're going to try this out, remember to run the upgrade External Method
as outlined in UPGRADE.txt.  Any and all feedback and suggestions would be
greatly appreciated.

Thanks!
_______________________

Ron Bickers
Logic Etc, Inc.