Apache Mod_rewrite question
Hi, I came across a strange behavior of mod_rewrite while using with zope. Although it is not a zope question, I was wondering if any of you have run into this. We are using mod_rewrite to redirect the users to one of the zope sites of a Zeo cluster. The directive RewriteMap has unexpected behavior when it is used to define more than one map with same map name in diffrent places. Normally all the directives within a tag applies only to that tag. For example: <VirtualHost *> ServerName www.foo.com DocumentRoot /var/www/foo RewriteEngine on RewriteLog /var/log/httpd/foo.log RewriteLogLevel 2 RewriteMap map rnd:/etc/httpd/conf/foomap.txt RewriteRule ^/Foo(.*) http://127.0.0.1${map:IP}/VirtualHostBase/http/foo/$1 [L,P] </VirtualHost> <VirtualHost *> ServerName www.bar.com DocumentRoot /var/www/bar RewriteEngine on RewriteLog /var/log/httpd/bar.log RewriteLogLevel 2 RewriteMap map rnd:/etc/httpd/conf/barmap.txt RewriteRule ^/Bar(.*) http://127.0.0.1${map:IP}/VirtualHostBase/http/bar/$1 [L,P] </VirtualHost> The above is expected to work. However ModRewrite, does not read the map file foomap.txt or barmap.txt everytime it needs it and it caches the results after the first time it reads for faster operation. The problem is the cache treats both the file as the same since they have the same name! Hence if you visit foo.com first, the data stored in foomap.txt is returned from the cache next time you visit foo.com or bar.com. This causes apache to redirect into the wrong zope site. Has anyone else experienced similar problems ? thanks, ganesh Zeomega LLC Open minds' Open Solutions Plano, Texas, USA Bangalore, India phone: 972-731-6750 http://www.zeomega.com
Hello Ganesha, Wednesday, July 14, 2004, 1:40:14 AM, you wrote: I'm using mod_rewrite with zope but without maps. Can your describe in details your task? GS> Hi, GS> I came across a strange behavior of mod_rewrite while using with zope. GS> Although it is not a zope question, I was wondering if any of you have GS> run into this. We are using mod_rewrite to redirect the users to one of GS> the zope sites of a Zeo cluster. GS> The directive RewriteMap has unexpected behavior when it is used to GS> define more than one map with same map name in diffrent places. Normally GS> all the directives within a tag applies only to that tag. GS> For example: GS> <VirtualHost *> GS> ServerName www.foo.com GS> DocumentRoot /var/www/foo GS> RewriteEngine on GS> RewriteLog /var/log/httpd/foo.log GS> RewriteLogLevel 2 GS> RewriteMap map rnd:/etc/httpd/conf/foomap.txt GS> RewriteRule ^/Foo(.*) GS> http://127.0.0.1${map:IP}/VirtualHostBase/http/foo/$1 [L,P] GS> </VirtualHost> GS> <VirtualHost *> GS> ServerName www.bar.com GS> DocumentRoot /var/www/bar GS> RewriteEngine on GS> RewriteLog /var/log/httpd/bar.log GS> RewriteLogLevel 2 GS> RewriteMap map rnd:/etc/httpd/conf/barmap.txt GS> RewriteRule ^/Bar(.*) GS> http://127.0.0.1${map:IP}/VirtualHostBase/http/bar/$1 [L,P] GS> </VirtualHost> GS> The above is expected to work. However ModRewrite, does not read the map GS> file foomap.txt or barmap.txt everytime it needs it and it caches the GS> results after the first time it reads for faster operation. GS> The problem is the cache treats both the file as the same since they GS> have the same name! Hence if you visit foo.com first, the data stored in GS> foomap.txt is returned from the cache next time you visit foo.com or GS> bar.com. This causes apache to redirect into the wrong zope site. GS> Has anyone else experienced similar problems ? GS> thanks, GS> ganesh GS> Zeomega LLC GS> Open minds' Open Solutions GS> Plano, Texas, USA GS> Bangalore, India GS> phone: 972-731-6750 GS> http://www.zeomega.com GS> _______________________________________________ GS> Zope-Dev maillist - Zope-Dev@zope.org GS> http://mail.zope.org/mailman/listinfo/zope-dev GS> ** No cross posts or HTML encoding! ** GS> (Related lists - GS> http://mail.zope.org/mailman/listinfo/zope-announce GS> http://mail.zope.org/mailman/listinfo/zope ) -- Best regards, Eugene mailto:el-spam@yandex.ru
Hi, We are using mod_rewrite to redirect the incoming requests to one of the zope instances running on the same machine. It is an SMP machine and has 4 zope instances. The map basically contains <IP> port1|port2|port3 Where port1 and port2 etc are the ports where the zope is running. The problem is mod_rewrite caches the map file based on map name instead of file as I have mentioned below. But we have solved the problem by using different map names in each of the <VirtualHost> sections. thanks, ganesh Zeomega LLC Open minds' Open Solutions Plano, Texas, USA Bangalore, India phone: 972-731-6750 http://www.zeomega.com On Wed, 2004-07-14 at 02:23, Eugene wrote:
Hello Ganesha, Wednesday, July 14, 2004, 1:40:14 AM, you wrote:
I'm using mod_rewrite with zope but without maps. Can your describe in details your task?
GS> Hi,
GS> I came across a strange behavior of mod_rewrite while using with zope. GS> Although it is not a zope question, I was wondering if any of you have GS> run into this. We are using mod_rewrite to redirect the users to one of GS> the zope sites of a Zeo cluster.
GS> The directive RewriteMap has unexpected behavior when it is used to GS> define more than one map with same map name in diffrent places. Normally GS> all the directives within a tag applies only to that tag.
GS> For example:
GS> <VirtualHost *> GS> ServerName www.foo.com GS> DocumentRoot /var/www/foo GS> RewriteEngine on GS> RewriteLog /var/log/httpd/foo.log GS> RewriteLogLevel 2 GS> RewriteMap map rnd:/etc/httpd/conf/foomap.txt GS> RewriteRule ^/Foo(.*) GS> http://127.0.0.1${map:IP}/VirtualHostBase/http/foo/$1 [L,P] GS> </VirtualHost>
GS> <VirtualHost *> GS> ServerName www.bar.com GS> DocumentRoot /var/www/bar GS> RewriteEngine on GS> RewriteLog /var/log/httpd/bar.log GS> RewriteLogLevel 2 GS> RewriteMap map rnd:/etc/httpd/conf/barmap.txt GS> RewriteRule ^/Bar(.*) GS> http://127.0.0.1${map:IP}/VirtualHostBase/http/bar/$1 [L,P] GS> </VirtualHost>
GS> The above is expected to work. However ModRewrite, does not read the map GS> file foomap.txt or barmap.txt everytime it needs it and it caches the GS> results after the first time it reads for faster operation.
GS> The problem is the cache treats both the file as the same since they GS> have the same name! Hence if you visit foo.com first, the data stored in GS> foomap.txt is returned from the cache next time you visit foo.com or GS> bar.com. This causes apache to redirect into the wrong zope site.
GS> Has anyone else experienced similar problems ?
GS> thanks, GS> ganesh
GS> Zeomega LLC GS> Open minds' Open Solutions
GS> Plano, Texas, USA GS> Bangalore, India GS> phone: 972-731-6750 GS> http://www.zeomega.com
GS> _______________________________________________ GS> Zope-Dev maillist - Zope-Dev@zope.org GS> http://mail.zope.org/mailman/listinfo/zope-dev GS> ** No cross posts or HTML encoding! ** GS> (Related lists - GS> http://mail.zope.org/mailman/listinfo/zope-announce GS> http://mail.zope.org/mailman/listinfo/zope )
Ganesha Shanmuganathan wrote at 2004-7-13 16:40 -0500:
... The directive RewriteMap has unexpected behavior when it is used to define more than one map with same map name in diffrent places. Normally all the directives within a tag applies only to that tag.
There are more strange things with Apache's "RewriteMap"s. Be warned: they are very inefficiently implemented for large maps! We used such maps to let Apache serve static CMF content as folkore reports that Apache is an order of magnitude faster than Zope in serving static content. It turned out that the solution using Apache RewriteMaps was slower than directly serving this content from Zope -- by a factor of 5! The explanation: Apache caches only single successful lookups in the map file. Each access to a key not in the cache results in parsing the map file until the key is found -- the whole map file when it does not contain the key. For larger map files, this has desastrous effects. Now, we use RewriteMaps only when the map files are very small (not more than some dozen entries).
... maps with the same name ... The problem is the cache treats both the file as the same since they have the same name! Hence if you visit foo.com first, the data stored in foomap.txt is returned from the cache next time you visit foo.com or bar.com. This causes apache to redirect into the wrong zope site.
We did not observe this behaviour... -- Dieter
Hi, On Wed, 2004-07-14 at 13:50, Dieter Maurer wrote:
Ganesha Shanmuganathan wrote at 2004-7-13 16:40 -0500:
... The directive RewriteMap has unexpected behavior when it is used to define more than one map with same map name in diffrent places. Normally all the directives within a tag applies only to that tag.
There are more strange things with Apache's "RewriteMap"s. Be warned: they are very inefficiently implemented for large maps!
We used such maps to let Apache serve static CMF content as folkore reports that Apache is an order of magnitude faster than Zope in serving static content.
It turned out that the solution using Apache RewriteMaps was slower than directly serving this content from Zope -- by a factor of 5!
The explanation: Apache caches only single successful lookups in the map file. Each access to a key not in the cache results in parsing the map file until the key is found -- the whole map file when it does not contain the key. For larger map files, this has desastrous effects.
Now, we use RewriteMaps only when the map files are very small (not more than some dozen entries). That is interesting to know. Right now, we use multiple map files but each is small. Is there anyother way to do load balncing with Zope without Rewrite ?
... maps with the same name ... The problem is the cache treats both the file as the same since they have the same name! Hence if you visit foo.com first, the data stored in foomap.txt is returned from the cache next time you visit foo.com or bar.com. This causes apache to redirect into the wrong zope site.
We did not observe this behaviour... btw, Did you use the same map name (like, in different sections maybe) for the different map files. That is what is causing it to fail. It is not a big issue but just wanted to confirm. Directives within one <Virtualhost> tag should not have any effect on others.
thanks, ganesh Zeomega LLC Open minds' Open Solutions Plano, Texas, USA Bangalore, India phone: 972-731-6750 http://www.zeomega.com
Ganesha Shanmuganathan wrote:
(..)Is there anyother way to do load balncing with Zope without Rewrite ?
Not sure, but would this help? http://www.backhand.org/mod_backhand/ regards jw -- Jan-Wijbrand Kolman jw@infrae.com
Ganesha Shanmuganathan wrote at 2004-7-14 17:26 -0500:
...
Now, we use RewriteMaps only when the map files are very small (not more than some dozen entries). That is interesting to know. Right now, we use multiple map files but each is small. Is there anyother way to do load balncing with Zope without Rewrite ?
We use a load balancer (a piece of hardware) for this. Alternatively, "pond" (or similar) is a software load balancer implemented in Python.
... maps with the same name ... The problem is the cache treats both the file as the same since they have the same name! Hence if you visit foo.com first, the data stored in foomap.txt is returned from the cache next time you visit foo.com or bar.com. This causes apache to redirect into the wrong zope site.
We did not observe this behaviour...
btw, Did you use the same map name (like, in different sections maybe) for the different map files.
Yes. The same map file name mapping to different files in different virtual hosts. -- Dieter
Hi, On Thu, 2004-07-15 at 15:28, Dieter Maurer wrote:
Ganesha Shanmuganathan wrote at 2004-7-14 17:26 -0500:
...
Now, we use RewriteMaps only when the map files are very small (not more than some dozen entries). That is interesting to know. Right now, we use multiple map files but each is small. Is there anyother way to do load balncing with Zope without Rewrite ?
We use a load balancer (a piece of hardware) for this.
Alternatively, "pond" (or similar) is a software load balancer implemented in Python.
Thanks . I will be sure to check out pond and backhand as well as mentioned by Jan-Wijbrand.
... maps with the same name ... The problem is the cache treats both the file as the same since they have the same name! Hence if you visit foo.com first, the data stored in foomap.txt is returned from the cache next time you visit foo.com or bar.com. This causes apache to redirect into the wrong zope site.
We did not observe this behaviour...
btw, Did you use the same map name (like, in different sections maybe) for the different map files.
Yes. The same map file name mapping to different files in different virtual hosts.
That is strange! It looks like a bug in mod_rewrite then. I checked the source and found that it does compare by map name. Thanks for your feedback ganesh Zeomega LLC Open minds' Open Solutions Plano, Texas, USA Bangalore, India phone: 972-731-6750 http://www.zeomega.com
Ganesha Shanmuganathan wrote at 2004-7-15 15:57 -0500:
... RewriteMap files with the same name are identified ... That is strange! It looks like a bug in mod_rewrite then. I checked the source and found that it does compare by map name. Thanks for your feedback
But the cache is not taken from a global variable but resolved via a local resource. I am not sure what this local resource is (I am no Apache expert). But, I expect that it is virtual host specific. -- Dieter
Hi Dieter, On Fri, 2004-07-16 at 13:10, Dieter Maurer wrote:
Ganesha Shanmuganathan wrote at 2004-7-15 15:57 -0500:
... RewriteMap files with the same name are identified ... That is strange! It looks like a bug in mod_rewrite then. I checked the source and found that it does compare by map name. Thanks for your feedback
But the cache is not taken from a global variable but resolved via a local resource.
I am not sure what this local resource is (I am no Apache expert). But, I expect that it is virtual host specific.
From what I understand, it is stored as static global (as APR table with APR mutex control for accessing by threads).
I changed it to use key,map_file_name and it works for me. I have a writeup at http://www.izpug.org/Members/zeomega/mod_rewrite, for more information in case you are interested. thanks, ganesh Zeomega LLC Open minds' Open Solutions Plano, Texas, USA Bangalore, India phone: 972-731-6750 http://www.zeomega.com
participants (4)
-
Dieter Maurer -
Eugene -
Ganesha Shanmuganathan -
Jan-Wijbrand Kolman