[Zope-CMF] Zope cache problems

Andy Bramah andybramah@cantab.net
Sun, 18 Aug 2002 22:26:18 +1100


Hello,

I've tried changing and removing the must-revalidate, and it doesn't seem to
make any difference. What should I be doing?

The caching problems seem to be really slowing my Plone site down. When I
request a Plone page from localhost, it loads almost instantaneously. But
when I try and access the site from elsewhere, the page takes ages to
appear. I use IE, and the blue progress bar moves quickly all the way over
to the right, then sits and does nothing for ages. The page then appears all
at once.

I notice this happens on www.plone.org as well.

Pages download almost instantly if I use the No CSS skin.

Any ideas?

Andy.

-----Original Message-----
From: zope-cmf-admin@zope.org [mailto:zope-cmf-admin@zope.org]On Behalf
Of Geoff Davis
Sent: 19 August 2002 05:24
To: Plone Developers; Zope CMF
Subject: [Zope-CMF] Zope cache problems


Hi all--

I was looking into the problems people were reporting about Plone CSS sheets
not caching.  These appear to be general ZServer issues, so I am
cross-posting to Zope CMF.

First, for some good background on cache issues, I read
http://www.web-caching.com/mnot_tutorial/
and skimmed section 13.3 of RFC2616:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13.3

Here's a greatly oversimplified summary of how caching works (at least in my
understanding):

Suppose a client requests an object.  The main types of caching that can
occur are as follows:

1) No caching.  When the client re-requests the object, the server sends it
back.

2) Browser caching.  The client gets the object from its local cache and
sends no request to the server.

3) Conditional browser caching.  When the client re-requests the object, it
sends the server a conditional GET that includes the last-modified time it
last saw for the object.  The server sees if its copy of the object has
changed since the browser's last-modified time.  If the object is still
fresh, the server sends a 304 but no object.  If the object is stale, the
server re-sends the object.  This is called weak validation.

4) Conditional browser caching.  When the client re-requests the object, it
sends the server a conditional GET that includes an Etag which, as far as I
can tell, is some kind of checksum on the object.  The server sees if its
copy of the object has a matching Etag.  If the object is still fresh, the
server sends a 304 but no object.  If the object is stale, the server
re-sends the object.  This is called strong validation.

(Note that my understanding of #4 might be off, and I don't understand how
validation requests are handled when Squid is between the client and
ZServer)

I used Web-Caching's cacheability tool (written in Python!) to see what was
going on with Plone:

http://www.web-caching.com/cgi-web-caching/cacheability.py?query=http%3A%2F%
2Fwww.plone.org&descend=on

Here's what I'm seeing:

* Images are getting sent out with no cache headers other than
Last-Modified.  It appears that they are being handled by case (3).
Requests for images are being sent each time, and the server returns a 304
but no image.  While that's good, the best case is (2) since ZServer still
has to process the request and there is still round-trip latency from
getting the 304 back to the client.

* CSS files set an "max-age=36000, must-revalidate" header.  What that is
supposed to do (I think) is tell the browser to do a conditional GET if it
has seen the CSS object in the last 10 hours and otherwise to do a regular
GET.  Here's the problem: when the browser does the conditional GET, the
server is sending a 200 and the entire object back.  That pretty much
defeats the whole caching setup.  The problem may be that ZServer is not
sending out any Etags; however, this is pretty much speculation on my part.

* There are page templates (personalize_form.pt and main_template.pt) that
set a "pragma no-cache" tag.  I don't think this is the best header to use.
Also, I believe that eventually the right way to set caching properties for
pt's may be to use the CMF Caching Policy Manager.  The policy manager has
some limitations on the kinds of files for which it works, and I don't fully
understand these.  It also doesn't give you complete control over the
headers, which is a real bummer.  However, it looks like for the class of
files for which it works, one can do interesting things like creating a
policy manager that will prevent, for example, page templates that end in
"_form" from being stored in a proxy cache.

Here are some fixes we should implement now:
(1)  Get rid of the must-revalidate in the CSS files.  If it is essential
for your CSS modifications to roll out immediately, change the name of the
CSS file.  One convention for doing this is to add the date/time of
modification to the file.  Since the CSS files are directly referenced in
only a few places, this is probably not a huge issue.

(2) I think the pragma no-cache header is suboptimal.  It's an HTTP 1.0
header, and doesn't take advantage of the much richer set of options
provided by HTTP 1.1.  I think it's worth having in the header to force
older HTTP 1.1 unaware proxy servers from doing inappropriate things, but I
think some additional HTTP 1.1 headers would help.  My understanding is that
when HTTP authentication is used (as it is on plone), pages are
automatically treated as cache-control: private.  That prevents pages from
being cached in shared caches (e.g. squid), but allows them to be cached
locally.  I think adding a cache-control:private tag to pages for which user
information is read from a cookie and inserted into a page (e.g. pages that
remember your login name) would be good.  That will prevent Squid from
caching the pages (bad), but will allow them to be cached locally (good).  I
need to do some more reading, though.

I am exploring some ideas for adding arbitrary headers to objects.

Geoff


Geoff Davis
http://www.geoffdavis.net


_______________________________________________
Zope-CMF maillist  -  Zope-CMF@zope.org
http://lists.zope.org/mailman/listinfo/zope-cmf

See http://collector.zope.org/CMF for bug reports and feature requests

__________________________________________________
Do You Yahoo!?
Everything you'll ever need on one web page
from News and Sport to Email and Music Charts
http://uk.my.yahoo.com