I have been trying to set up Squid as an httpd accelerator for Zope with the idea of speeding up load times, especially for Plone. Unfortunately Squid appears to simply redirect any requests rather than serving them from cache. Is there anything I can do within Zope to allow pages to be cached? Or have I overlooked a Squid setting? -- John
John Poltorak wrote:
I have been trying to set up Squid as an httpd accelerator for Zope with the idea of speeding up load times, especially for Plone. Unfortunately Squid appears to simply redirect any requests rather than serving them from cache. Is there anything I can do within Zope to allow pages to be cached? Or have I overlooked a Squid setting?
You need to set cache headers. I use the cache policy manager in the CMF for this. Note: If Squid and Zope are on different servers, make sure your clocks are in sync..... Andrew -- Zope Managed Hosting Systems Administrator/Software Engineer Zope Corporation (540) 361-1700
On Mon, Dec 13, 2004 at 02:15:01PM -0500, Andrew Sawyers wrote:
John Poltorak wrote:
I have been trying to set up Squid as an httpd accelerator for Zope with the idea of speeding up load times, especially for Plone. Unfortunately Squid appears to simply redirect any requests rather than serving them from cache. Is there anything I can do within Zope to allow pages to be cached? Or have I overlooked a Squid setting?
You need to set cache headers. I use the cache policy manager in the CMF for this. Note: If Squid and Zope are on different servers, make sure your clocks are in sync.....
I'm using the same host... Is the cache policy manager something I should expect to be installed already? Where do I read up about it?
Andrew
-- Zope Managed Hosting Systems Administrator/Software Engineer Zope Corporation (540) 361-1700
-- John
John Poltorak wrote:
On Mon, Dec 13, 2004 at 02:15:01PM -0500, Andrew Sawyers wrote:
John Poltorak wrote:
I have been trying to set up Squid as an httpd accelerator for Zope with the idea of speeding up load times, especially for Plone. Unfortunately Squid appears to simply redirect any requests rather than serving them from cache. Is there anything I can do within Zope to allow pages to be cached? Or have I overlooked a Squid setting?
You need to set cache headers. I use the cache policy manager in the CMF for this. Note: If Squid and Zope are on different servers, make sure your clocks are in sync.....
I'm using the same host... Is the cache policy manager something I should expect to be installed already? Not sure; I typically script all my builds, so I do it manually. In your CMF instance, it will be in the ZMI product pull-down: "CMF Caching Policy Manager"
Where do I read up about it?
Good question :) I don't know.... For a test, you can setup a Policy as follows: Policy ID: default Predicate: python:None Max age: 300 <-- 5 minutes Check the 'Must-Revalidate' box. Depending on your expectations and needs, it could get complex setting up policies. Andrew -- Zope Managed Hosting Systems Administrator/Software Engineer Zope Corporation (540) 361-1700
On Mon, Dec 13, 2004 at 02:30:42PM -0500, Andrew Sawyers wrote:
John Poltorak wrote:
On Mon, Dec 13, 2004 at 02:15:01PM -0500, Andrew Sawyers wrote:
John Poltorak wrote:
I have been trying to set up Squid as an httpd accelerator for Zope with the idea of speeding up load times, especially for Plone. Unfortunately Squid appears to simply redirect any requests rather than serving them from cache. Is there anything I can do within Zope to allow pages to be cached? Or have I overlooked a Squid setting?
You need to set cache headers. I use the cache policy manager in the CMF for this. Note: If Squid and Zope are on different servers, make sure your clocks are in sync.....
I'm using the same host... Is the cache policy manager something I should expect to be installed already? Not sure; I typically script all my builds, so I do it manually. In your CMF instance, it will be in the ZMI product pull-down: "CMF Caching Policy Manager"
This is something completely new to me... Do I install a CMP for each site I host or can I have a global policy for the Zope instance?
Andrew
-- Zope Managed Hosting Systems Administrator/Software Engineer Zope Corporation (540) 361-1700
-- John
John Poltorak wrote:
This is something completely new to me... Do I install a CMP for each site I host or can I have a global policy for the Zope instance?
You may do either. CMF skin methods will acquire 'caching_policy_manager', if they can, from any parent (but only from the "nearest" one above them). Tres. -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com
On Mon, Dec 13, 2004 at 06:32:55PM -0500, Tres Seaver wrote:
John Poltorak wrote:
This is something completely new to me... Do I install a CMP for each site I host or can I have a global policy for the Zope instance?
You may do either. CMF skin methods will acquire 'caching_policy_manager', if they can, from any parent (but only from the "nearest" one above them).
How can I see the effects of the caching policy manager? Does it create new headers? I've added a CPM but can't tell if it is doing anything.
Tres. -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com
-- John
John Poltorak wrote:
How can I see the effects of the caching policy manager? Does it create new headers? I've added a CPM but can't tell if it is doing anything.
As its name might indicate, a CPM holds a set of caching policies. Each policy consists of a predicate (defined as a TALES expression returning a boolean) and a set of properties which define the headers set by the policy. For a given request, the first policy whose predicate returns True gets to set the headers. Andy Mckay's Plone book has some useful pointers on using CPM: http://his.biologie.hu-berlin.de/PloneBook/ch14.rst#caching-content-types And I think Andrew suggested a testable default policy earlier in the thread. Tres. -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com
John Poltorak wrote:
On Mon, Dec 13, 2004 at 06:32:55PM -0500, Tres Seaver wrote:
John Poltorak wrote:
This is something completely new to me... Do I install a CMP for each site I host or can I have a global policy for the Zope instance?
You may do either. CMF skin methods will acquire 'caching_policy_manager', if they can, from any parent (but only from the "nearest" one above them).
How can I see the effects of the caching policy manager? Does it create new headers? I've added a CPM but can't tell if it is doing anything.
Tres. -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com
What policy did you add? If you followed my example, look for the cache-control header. Use wget -S url Andrew -- Zope Managed Hosting Systems Administrator/Software Engineer Zope Corporation (540) 361-1700
On Tue, Dec 14, 2004 at 10:13:37AM -0500, Andrew Sawyers wrote:
John Poltorak wrote:
On Mon, Dec 13, 2004 at 06:32:55PM -0500, Tres Seaver wrote:
John Poltorak wrote:
This is something completely new to me... Do I install a CMP for each site I host or can I have a global policy for the Zope instance?
You may do either. CMF skin methods will acquire 'caching_policy_manager', if they can, from any parent (but only from the "nearest" one above them).
How can I see the effects of the caching policy manager? Does it create new headers? I've added a CPM but can't tell if it is doing anything.
Tres. -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com
What policy did you add?
In the root folder I have:- Policy ID policy1 Predicate python:1 Mod.Time content/modified Max age (secs) 300 some of these values were defaults.
If you followed my example, look for the cache-control header.
Use wget -S url
wget -S http://localhost/ returns:- --15:22:01-- http://localhost/ => `index.html.1' Resolving localhost... done. Connecting to localhost[127.0.0.1]:80... connected. HTTP request sent, awaiting response... 1 HTTP/1.0 200 OK 2 Server: Zope/(Zope 2.7.3-0, python 2.3.4, os2emx) ZServer/1.1 Plone/2.0.5 3 Date: Tue, 14 Dec 2004 15:22:02 GMT 4 Content-Length: 3053 5 Content-Type: text/html 6 X-Cache: MISS from localhost 7 X-Cache-Lookup: HIT from localhost:80 8 Connection: keep-alive Not sure if this includes the cache-control header you are referring to... Should squidclient return something similar?
Andrew
-- Zope Managed Hosting Systems Administrator/Software Engineer Zope Corporation (540) 361-1700
-- John
On Tue, Dec 14, 2004 at 03:28:49PM +0000, John Poltorak wrote:
Not sure if this includes the cache-control header you are referring to...
I suggest that you have a look at Mark Nottingham's "Caching Tutorial", <http://www.mnot.net/cache_docs/>. See also his Cacheability Engine which "Examines Web pages to determine how they will interact with Web caches". -- Fred Yankowski fred@ontosys.com tel: +1.630.879.1312 OntoSys, Inc PGP keyID: 7B449345 fax: +1.630.879.1370 www.ontosys.com 38W242 Deerpath Rd, Batavia, IL 60510-9461, USA
On Tue, Dec 14, 2004 at 10:02:51AM -0600, Fred Yankowski wrote:
On Tue, Dec 14, 2004 at 03:28:49PM +0000, John Poltorak wrote:
Not sure if this includes the cache-control header you are referring to...
I suggest that you have a look at Mark Nottingham's "Caching Tutorial", <http://www.mnot.net/cache_docs/>. See also his Cacheability Engine which "Examines Web pages to determine how they will interact with Web caches".
Many thanks for the caching tutorial - it looks pretty useful, but I'm not clear about what sets the cache-control headers... I'm assuming that CMF's Caching Policy Manager sets those and since I have a single policy which sets Max age to 300 I would have expect to see a header which mentioned cache-control and max-age, but I only see these two:- X-Cache: MISS from localhost X-Cache-Lookup: MISS from localhost:80 which I think must be provided by Squid. Looks like I've overlooked something...
-- Fred Yankowski fred@ontosys.com tel: +1.630.879.1312 OntoSys, Inc PGP keyID: 7B449345 fax: +1.630.879.1370 www.ontosys.com 38W242 Deerpath Rd, Batavia, IL 60510-9461, USA
-- John
John Poltorak wrote:
On Tue, Dec 14, 2004 at 10:02:51AM -0600, Fred Yankowski wrote:
On Tue, Dec 14, 2004 at 03:28:49PM +0000, John Poltorak wrote:
Not sure if this includes the cache-control header you are referring to...
I suggest that you have a look at Mark Nottingham's "Caching Tutorial", <http://www.mnot.net/cache_docs/>. See also his Cacheability Engine which "Examines Web pages to determine how they will interact with Web caches".
Many thanks for the caching tutorial - it looks pretty useful, but I'm not clear about what sets the cache-control headers... I'm assuming that CMF's Caching Policy Manager sets those and since I have a single policy which sets Max age to 300 I would have expect to see a header which mentioned cache-control and max-age, but I only see these two:-
X-Cache: MISS from localhost X-Cache-Lookup: MISS from localhost:80
which I think must be provided by Squid. Looks like I've overlooked something...
-- Fred Yankowski fred@ontosys.com tel: +1.630.879.1312 OntoSys, Inc PGP keyID: 7B449345 fax: +1.630.879.1370 www.ontosys.com 38W242 Deerpath Rd, Batavia, IL 60510-9461, USA
I don't know off the top of my head, but the CMF Cache Policy Manager doesn't "just" work with everything. I'd have to look and see, but there are expectations which must be met for it to get called and the headers get injected. My last email shows another way to get a test header so you can see it working. At the end of the day, the you will need to come up with a policy. I typically have several policies defined in my caching policy manager. Over several projects I've utilized the CMF CPM and the Accelerated HTTP Cache Manager which is just a Zope product. Andrew -- Zope Managed Hosting Systems Administrator/Software Engineer Zope Corporation (540) 361-1700
In article <20041214162212.K47@warpix.org> you write:
On Tue, Dec 14, 2004 at 10:02:51AM -0600, Fred Yankowski wrote:
On Tue, Dec 14, 2004 at 03:28:49PM +0000, John Poltorak wrote:
Not sure if this includes the cache-control header you are referring to...
I suggest that you have a look at Mark Nottingham's "Caching Tutorial", <http://www.mnot.net/cache_docs/>. See also his Cacheability Engine which "Examines Web pages to determine how they will interact with Web caches".
Many thanks for the caching tutorial - it looks pretty useful, but I'm not clear about what sets the cache-control headers... I'm assuming that CMF's Caching Policy Manager sets those and since I have a single policy which sets Max age to 300 I would have expect to see a header which mentioned cache-control and max-age, but I only see these two:-
X-Cache: MISS from localhost X-Cache-Lookup: MISS from localhost:80
which I think must be provided by Squid. Looks like I've overlooked something...
Try to talk to the Zope directly, without Squid, to see what headers it sends. Or do a network trace. Florent -- Florent Guillaume, Nuxeo (Paris, France) CTO, Director of R&D +33 1 40 33 71 59 http://nuxeo.com fg@nuxeo.com
John Poltorak wrote:
On Tue, Dec 14, 2004 at 10:13:37AM -0500, Andrew Sawyers wrote:
John Poltorak wrote:
On Mon, Dec 13, 2004 at 06:32:55PM -0500, Tres Seaver wrote:
John Poltorak wrote:
This is something completely new to me... Do I install a CMP for each site I host or can I have a global policy for the Zope instance?
You may do either. CMF skin methods will acquire 'caching_policy_manager', if they can, from any parent (but only from the "nearest" one above them).
How can I see the effects of the caching policy manager? Does it create new headers? I've added a CPM but can't tell if it is doing anything.
Tres. -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com
What policy did you add?
In the root folder I have:-
Policy ID policy1 Predicate python:1 Mod.Time content/modified Max age (secs) 300
some of these values were defaults.
If you followed my example, look for the cache-control header.
Use wget -S url
wget -S http://localhost/ returns:-
--15:22:01-- http://localhost/ => `index.html.1' Resolving localhost... done. Connecting to localhost[127.0.0.1]:80... connected. HTTP request sent, awaiting response... 1 HTTP/1.0 200 OK 2 Server: Zope/(Zope 2.7.3-0, python 2.3.4, os2emx) ZServer/1.1 Plone/2.0.5 3 Date: Tue, 14 Dec 2004 15:22:02 GMT 4 Content-Length: 3053 5 Content-Type: text/html 6 X-Cache: MISS from localhost 7 X-Cache-Lookup: HIT from localhost:80 8 Connection: keep-alive
Not sure if this includes the cache-control header you are referring to...
Should squidclient return something similar?
Andrew
-- Zope Managed Hosting Systems Administrator/Software Engineer Zope Corporation (540) 361-1700
So, the cache header has not gotten in. Are you using a file system skin to render the site? I see it's Plone/2.0.5 -- I know 0 about Plone. So, test this: Add a python script to your site's main_template, something like: <div tal:define="cache_header here/cache_header"/> where cache_header is a python script: Create a python script in the root of your site: cache_header.py context.REQUEST.RESPONSE.setHeader('Cache-Control','max-age=30, must-revalidate') Then you should see the caching headers in your Zope responses and the pages be cached in your Squid. Note, unless you want every page cached with a max-age=value as the above python script would do - this is not the *right* method. I'm just trying to help you get rolling. If all requests can be cached for a specific amount of time - then you can accomplish this completely in squid and not require the Zope server to set any headers. What is your cache strategy? I'm just guessing, but I'd thought you were just getting rolling and learning. While this isn't really hard - it's not "free" and will take some thought, design, and setup if it's not a simple cache policy you're after. Andrew -- Zope Managed Hosting Systems Administrator/Software Engineer Zope Corporation (540) 361-1700
John Poltorak wrote:
I have been trying to set up Squid as an httpd accelerator for Zope with the idea of speeding up load times, especially for Plone. Unfortunately Squid appears to simply redirect any requests rather than serving them from cache. Is there anything I can do within Zope to allow pages to be cached? Or have I overlooked a Squid setting?
John, I don't know about cache management in Zope (per other responses to your post), but make sure you've got Squid configured correctly as well. For example, I found that I had to use some explicit refresh patterns in squid.conf (I don't know why), such as: # Folders/default pages # These don't seem to get cached without a refresh pattern refresh_pattern /$ 30 50% 60 Use the debug options in Squid. I also recommend looking at the actual HTTP headers being sent/received. I use LiveHTTPHeaders for Mozilla/Firefox (http://livehttpheaders.mozdev.org/). --David
On Mon, Dec 13, 2004 at 04:45:44PM -0500, David Chandek-Stark wrote:
John Poltorak wrote:
I have been trying to set up Squid as an httpd accelerator for Zope with the idea of speeding up load times, especially for Plone. Unfortunately Squid appears to simply redirect any requests rather than serving them from cache. Is there anything I can do within Zope to allow pages to be cached? Or have I overlooked a Squid setting?
John, I don't know about cache management in Zope (per other responses to your post), but make sure you've got Squid configured correctly as well. For example, I found that I had to use some explicit refresh patterns in squid.conf (I don't know why), such as:
# Folders/default pages # These don't seem to get cached without a refresh pattern refresh_pattern /$ 30 50% 60
Use the debug options in Squid. I also recommend looking at the actual HTTP headers being sent/received. I use LiveHTTPHeaders for Mozilla/Firefox (http://livehttpheaders.mozdev.org/).
There certainly seems a lot more to setting up an httpd accelerator than alluded to in the Squid FAQ....
--David
-- John
On Mon, Dec 13, 2004 at 04:45:44PM -0500, David Chandek-Stark wrote:
John Poltorak wrote:
I have been trying to set up Squid as an httpd accelerator for Zope with the idea of speeding up load times, especially for Plone. Unfortunately Squid appears to simply redirect any requests rather than serving them from cache. Is there anything I can do within Zope to allow pages to be cached? Or have I overlooked a Squid setting?
John, I don't know about cache management in Zope (per other responses to your post), but make sure you've got Squid configured correctly as well. For example, I found that I had to use some explicit refresh patterns in squid.conf (I don't know why), such as:
# Folders/default pages # These don't seem to get cached without a refresh pattern refresh_pattern /$ 30 50% 60
Can you explain that? I put it in but did see any improvement.
Use the debug options in Squid. I also recommend looking at the actual HTTP headers being sent/received. I use LiveHTTPHeaders for Mozilla/Firefox (http://livehttpheaders.mozdev.org/).
I had never come across LiveHTTPHeaders before and it does look useful, but since installing the product Firefox won't start up any more :-(... Need to work out how to uninstall it without starting up Firefox.
--David
-- John
participants (6)
-
Andrew Sawyers -
David Chandek-Stark -
Florent Guillaume -
Fred Yankowski -
John Poltorak -
Tres Seaver