[Zope-CMF] Running Zope behind Squid on the same host

Chris McDonough chrism@zope.com
30 Jan 2003 19:18:45 -0500


I don't think redirector speed is a problem in practice.  Python can do
this job (probably on the order of thousands of redirector rewrites per
second) much faster than a Squid can service requests (~ 400 - 700 req/s
for an average site on average hardware).

On Thu, 2003-01-30 at 17:41, sean.upton@uniontrib.com wrote:
> Thanks for the write-up on this.  As a fellow Squid+Zope user, I think this
> is helpful.  I haven't used SquidGuard, though I considered it at one point;
> I might also recommend considering using Pyredir
> (http://freshmeat.net/projects/pyredir/) as your redirector instead of
> Squidguard; it is written in Python (thus can be easily hacked, for example,
> I turned off logging), and has a simple configuration file syntax (one line
> per rule, an example is below).
> 
> #============= Top part of my pyredir.conf file =======
> methods: GET,POST,HEAD,ICP_QUERY
> clients: 0.0.0.0/0
> ^http://classifieds.uniontrib.com[/]?(.*)
> =http://nodes:9673/Classifieds/VirtualHostBase/http/classifieds.uniontrib.co
> m:80/Classifieds/VirtualHostRoot/\1
> #======================================================
> 
> I haven't noticed there being any issues with speed, though this is likely
> to be slower than a redirector written in C.  For an accelerator, it would
> be nice to minimize this, by optionally caching redirection values (thus
> saving Squid the time needed for communication between the redirector
> process and itself, as well as the CPU time needed by the redirector);
> unfortunately, Squid, at the moment does not cache redirector return values,
> though one could likely take the code in Squid for IP lookup caching and use
> the same idea to cache the returned resulting URLs passed from the
> redirector.  I may look to do this in the future, one day when I have time.
> 
> Sean
> 
> -----Original Message-----
> From: J C Lawrence [mailto:claw@kanga.nu]
> Sent: Wednesday, January 29, 2003 12:50 PM
> To: zope-cmf@zope.org
> Subject: [Zope-CMF] Running Zope behind Squid on the same host
> 
> 
> 
> The typical advice is to run Zope behind Apache.  For various reasons,
> among which are that I found Squid faster, I don't want to do that.  I
> went thru the HOWTOs on zope.org and found them ineffective.  In
> particular Marc Bowery's HOWTO at:
> 
>   http://www.zope.org/Members/bowerymarc/squid-zserver-virtual
> 
> cost me two days before I abandoned it as a dead end and not quite what
> I needed.  The below approach required little more than 45 minutes to
> discover, research, and implement.
> 
>   Note: I've not (yet) paid mind to implementing proper Squid cache
>   security so as to not be operating an open cache for others to abuse.
>   As such I won't be addressing those concerns.  However, they are not
>   difficult problems and there are thorough and articulate discussions
>   of the area for Squid as a general product, outside of any Zope
>   specifics.  As such, I'll leave those discussions there.
> 
>   You don't want to be running an open cache any more than you want to
>   be running an open mail relay.  Pay attention.  You have been warned.
> 
> Here's how I got it working.
> 
>   Notes:
> 
>     - Nothing here is CMF specific.  However, I'm not on the base Zope
>     list and I am on this list.  Please feel free to crosspost this to
>     the base Zope lists as you see fit.
> 
>     - Some details are mildly Linux/Debian specific, butt they should be
>     obvious and easy to translate for other Linux distributions or other
>     Unixes.
> 
>     - Please forward corrections, suggestions, or updates to me at the
>     above address and I'll look to incorporating them into this doc.
> 
> Tools needed:
> 
>   - Zope               -- http://www.zope.org
> 
>   - Squid              -- http://www.squid-cache.org/
> 
>   - SquidGuard         -- http://www.squidguard.org/
> 
>   - SiteAccessEnhanced -- http://www.zope.org/Members/sfm/SiteAccessEnhanced
> 
>   *Note: I initially tried to use Jesred but was unable to get it to
>   process http POSTs correctly and reliably.  While it is simpler and
>   smaller than SquidGuard, it also appears to be slower.*
> 
> Installation:
> 
>   Install all the above in the normal way.  For Linux/Debian Zope,
>   Squid, and SquidGuard and be retrieved and installed via `apt-get`.
>   You'll need to install SiteAccessEnhanced in the normal Zope way by
>   unpacking it in your Zope products directory and restarting Zope.
> 
> Configuration:
> 
>   If you have any questions on the below, please consult the relevant
>   documentation, help files, web sites, and sources before asking me.
> 
>   - Zope:
> 
>     Using the ZMI create a VirtualHostMonster (the name of the product
>     that SiteAccessEnhanced installs) in the root of your Zope system.
>     Call it anything you want.
> 
>   - Squid:
> 
>     Use the following configuration options over and above the Debian
>     defaults:
> 
>       http_port 80
>       redirect_program /usr/bin/squidGuard -c /etc/squid/squidGuard.conf
>       httpd_accel_host virtual
>       httpd_accel_port 0
> 
>     Other changes may be necessary if you are not using Linux/Debian or
>     have a different initial default squid.conf.
> 
>   - SquidGuard:
> 
>     Use the following SquidGuard configuration file::
> 
>        dbhome /var/lib/squidguard/db
>        logdir /var/log/squid
>        acl {
>            default {
>                redirect
> http://localhost:9673/VirtualHostBase/http/your.host.dom:80/root_folder/Virt
> ualHostRoot/%p
>             }
>        }
> 
>     Where:
> 
>       your.host.dom -- is the FQDN of the host in question.
> 
>       root_folder -- is the Zope folder that you want to be the root
>       folder of your site.
> 
>       9673 -- Is the default port that Linux/Debian runs Zope on.
>       Change to suit your installation.
> 
>     You could also use rewrite rules under SquidGuard instead of the
>     redirect used above, but they are more expensive.
> 
>   - SiteAccessEnhanced
> 
>     Using the ZMI install a VirtualHostMonster object in the root folder
>     of your Zope installatiom.  You will not need to configure it for
>     this simple HOWTO.
> 
> Voila!  Restart Zope and Squid for the changes to take effect and
> everything should be happily working on port 80.  
> 
> Note that you'll need to do something more complex and interesting for
> proper virtual host support.  As this is a simple get-you-started HOWTO
> I've not covered that.
> 
> I've also posted this HOWTO to Zope.org at:
> 
>   http://www.zope.org/Members/JCLawrence/LocalhostSquidHOWTO/index_html
> 
> and will be the current version at any time.
> 
> Enjoy.
> 
> --
> J C Lawrence
> ---------(*)                Satan, oscillate my metallic sonatas.
> claw@kanga.nu               He lived as a devil, eh?
> http://www.kanga.nu/~claw/  Evil is a name of a foeman, as I live.
> 
> 
> _______________________________________________
> Zope-CMF maillist  -  Zope-CMF@zope.org
> http://lists.zope.org/mailman/listinfo/zope-cmf
> 
> See http://collector.zope.org/CMF for bug reports and feature requests
> 
> _______________________________________________
> Zope-CMF maillist  -  Zope-CMF@zope.org
> http://mail.zope.org/mailman/listinfo/zope-cmf
> 
> See http://collector.zope.org/CMF for bug reports and feature requests