[Zope-CMF] Zope cache problems

[Geoff Davis]

I have done some more experiments and have found that CSS caching in Plone
works, but not completely, under the current setup.

What I did:

* I created a new plone site on my local server and then navigated around in
it.  The whole time I tracked my ZServer log to see what was getting
requested and what was getting returned.

What works:

* When I make an initial request, a whole bunch of stuff loads -- the main
page, a ton of style sheets, and a few images.
* When I navigate around, only the page templates load -- the style sheets
and images are pulled from my browser cache.

What doesn't work:

* When I hit the RELOAD button on my browser, the browser does a conditional
GET, which triggers validation on all objects.  Images correctly return
304's (no change), but style sheets and page templates return 200's (they
are completely re-sent).

At first this behavior seems broken -- after all, it's really wasteful to
re-send all the style sheets.  I decided to poke around the ZServer source
to figure out what was going on.

The problem:

* The Plone CSS files are all generated by DTML scripts.  When ZServer gets
a conditional GET for something produced by a script, it can't just use the
Last-Modified time to perform validation, because scripts produce dynamic
content -- the Last-Modified time for a script is insufficient to tell you
whether the content of the script has changed.  If ZServer implemented
ETags, it could conceivably render the page locally, generate an MD5
signature, and send it along as an ETag.  Validation could trigger a
re-generation of the page and the resulting MD5 tag, which could then be
used to determine freshness.  Such a scheme could result in a small
reduction in network traffic, but it would probably have a big negative
impact on the server load because of all the MD5 calls.

Some ideas:

I don't understand how ZServer works well enough to figure out how to trick
it into thinking it is serving up static content (and I'm not sure that
would be a good idea in any case).

One possibility would be to run Zope behind Apache and have Apache serve up
the static content.  Apache has an easy-to-use module to allow one to set
expiration headers on the content it serves (mod_expires).  It would be nice
to be able to set image expiration headers explicitly rather than relying on
whatever mysterious cache strategies browsers use.

Another possibility would be to run Zope behind Squid.  I'm not sure how
Squid handles the validation issue -- one important experiment would be to
see if Squid passes along validation requests to ZServer.  If it does, that
could be a problem.

A third possibility would be to create a portal tool called, say,
"static_content", that sets object expiration headers.  The "static_content"
tool would look like a folder, though it would not contain anything.
Anything referenced in the folder via a URL of the form
".../static_content/my_image.gif" or ".../static_content/my_style.css" would
be returned with expiration headers set.  That might let us change many
(most?) of the Plone CSS sheets back to static content for which ZServer
will do proper validation, though I'm not certain.  I'm going to do some
experiments soon to test the feasibility of such a thing.


Geoff Davis
http://www.geoffdavis.net


----- Original Message -----
From: "Andy Bramah" <andybramah@cantab.net>
To: "Geoff Davis" <geoff@geoffdavis.net>; "Andy Bramah"
<andybramah@cantab.net>; "Plone Developers"
<plone-developers@lists.sourceforge.net>; "Zope CMF" <zope-cmf@zope.org>
Sent: Sunday, August 18, 2002 7:50 AM
Subject: RE: [Zope-CMF] Zope cache problems


> I've got this at the top of the css content
>
> <dtml-let last_modified="_.DateTime()-14"
> expires="_.DateTime()+7" >
> <dtml-call "REQUEST.RESPONSE.setHeader( 'Content-Type', 'text/css' )">
> <dtml-call "REQUEST.RESPONSE.setHeader( 'Last-Modified',
> last_modified.toZone('GMT').rfc822() )">
> <dtml-call "REQUEST.RESPONSE.setHeader( 'Cache-Control', 'max-age=36000,
> must-revalidate' )">
> <dtml-call "REQUEST.RESPONSE.setHeader( 'Expires',
> expires.toZone('GMT').rfc822() )" >
> </dtml-let>
>
> This is what Plone has out of the box. Tried removing the Cache-Control
> line - didn't work, and the tool you recommend reports the same error (the
> object is returned when only validation is requested).
>
> I'm using Zope 2.5.1, CMF1.3b2, Plone1.0a. Maybe I'm being dumn and should
> try updating to something later - only I've looked on CVS and it uses the
> same header I've pasted above.
>
> Also, which lists should I be replying to? I don't want to annoy anyone
with
> my continuing cache problems...!
>
> Andy.
>