Regular expressions should be allowed by default. I've spent some time trying to find out why regular expressions are not allowed in Zope throught-the-web development. The answer I hear is: "Because it's a security issue". Digging a little deeper, it turns out to be because TTW script developers can cause a Denial of Service from Zope by writing a particular nasty regular expression in a script, causing Zope to use 100% cpu time. So it's a question of trust. But surely a script developer can be trusted not to cause a DoS on the site he's working on! Script developers should be empowered, not crippled! Some common objections: "People can just write an external scripts instead." - True, but it also makes everyday work much more cumbersome. And some script developers don't have access to the file system. Surely, TTW scripts exist to make life easier for site developers. "The administrator can allow the 're' module." Also true, but some people won't do that because they think it will expose their site to all kind of attacks from anonymous users. This isn't the case (or is it?). So there it is. I'm writing this because I think that Zope is missing out on a great feature, and because I haven't gotten any answers indicating that there are other (worse) reasons why regular expressions are banned. Am I wrong? Am I being silly here? Sincerely, Tue Wennerberg Civilingeniør og Freelance Udvikler http://tuewennerberg.dk/ - tue@wennerberg.dk - (+45) 4043 6735
At 10:17 AM 1/19/2003, Tue Wennerberg wrote:
So it's a question of trust. But surely a script developer can be trusted not to cause a DoS on the site he's working on! Script developers should be empowered, not crippled!
Zope empowers the admin to control the environment. If the admin trusts the developers, that trust can be extended... but it is appropriate that someone with command-line access should be involved any time a developer wants to run unrestricted code. I think it's a question of preferring a configuration that is "more secure" by default and giving admins full power to loosen restrictions as they see fit.
And some script developers don't have access to the file system.
That's exactly the point.
So there it is. I'm writing this because I think that Zope is missing out on a great feature, and because I haven't gotten any answers indicating that there are other (worse) reasons why regular expressions are banned. Am I wrong? Am I being silly here?
It might be fun and/or interesting to make a product that validates and performs regexes in a trustworthy fashion. I'm not a regex guru, so I'm not sure exactly what level of validation is involved here. Installing something like this would still require admin participation, but could hook into the existing access controls such that use of the product could be restricted on a per-developer basis. Just thinkin'... Dylan
Tue Wennerberg wrote:
Regular expressions should be allowed by default.
I've spent some time trying to find out why regular expressions are not allowed in Zope throught-the-web development.
The answer I hear is: "Because it's a security issue". Digging a little deeper, it turns out to be because TTW script developers can cause a Denial of Service from Zope by writing a particular nasty regular expression in a script, causing Zope to use 100% cpu time.
Well, I vaguely remember having participated in a discussion about that also, and I believe I chipped in an example like: for a in range(0,1000): for b in range(0,1000): for c in range(0,1000): ... you get the picture. Thus demonstrating that a malicious scripter could always cause a DOS. The answer I got, IIRC, was that the point is not to guard against maliciousness, but against stupidity. And if you think about it, it's quite more likely someone writing a working, but extremely bad performing regexp, which kills the server as soon as put into production use, than someone accidently nesting loops like I wrote above. I don't know about you, but I think this argument is at least more convincing than the "malicious scripter" one. After all, the absence of strcpy() in python is a feature, isn't it? ;) cheers, oliver
Oliver Bleutgen wrote:
Tue Wennerberg wrote:
Regular expressions should be allowed by default.
I've spent some time trying to find out why regular expressions are not allowed in Zope throught-the-web development.
The answer I hear is: "Because it's a security issue". Digging a little deeper, it turns out to be because TTW script developers can cause a Denial of Service from Zope by writing a particular nasty regular expression in a script, causing Zope to use 100% cpu time.
Well, I vaguely remember having participated in a discussion about that also, and I believe I chipped in an example like:
for a in range(0,1000): for b in range(0,1000): for c in range(0,1000):
... you get the picture.
Thus demonstrating that a malicious scripter could always cause a DOS.
My point exactly! The Zope book actually states: "Loop limits Scripts cannot create infinite loops. If your script loops a very large number of times Zope will raise an error. This restriction covers all kinds of loops including for and while loops. The reason or this restriction is to limit your ability to hang Zope by creating an infinite loop." But I'm pretty sure there's always a way around this. Probably your example is one. Point is, you should be able to trust your script developer.
The answer I got, IIRC, was that the point is not to guard against maliciousness, but against stupidity. And if you think about it, it's quite more likely someone writing a working, but extremely bad performing regexp, which kills the server as soon as put into production use, than someone accidently nesting loops like I wrote above. I don't know about you, but I think this argument is at least more convincing than the "malicious scripter" one.
Well, now we're getting somewhere. I believe that "guarding against stupidity" is a much more valid point. However, still not valid enough that regular expressions should be banned, since regular expressions would be such a great feature for Zope. In my eyes, a script developer should be trusted to create well-written code. In other words, badly developed scripts cause a badly developed site, which shouldn't surprise anyone. I don't think Zope should (or can) protect against stupidity. In my experience, when non-expert developers create regular expressions, they are always trivial expressions, which don't cause such problems. Of course a programming error shouldn't be able to shutdown an entire system, but that should be solved in another way (e.g. resource control for individual processes/threads). -- Mvh. Tue Wennerberg Civilingeniør og Freelance Udvikler http://tuewennerberg.dk/ - tue@wennerberg.dk - (+45) 4043 6735
Tue Wennerberg wrote:
Well, now we're getting somewhere. I believe that "guarding against stupidity" is a much more valid point. However, still not valid enough that regular expressions should be banned, since regular expressions would be such a great feature for Zope.
It's not as you couldn't use regexps in zope, it's just not as easy as you like it to be.
In my eyes, a script developer should be trusted to create well-written code. In other words, badly developed scripts cause a badly developed site, which shouldn't surprise anyone. I don't think Zope should (or can) protect against stupidity. In my experience, when non-expert developers create regular expressions, they are always trivial expressions, which don't cause such problems.
Of course a programming error shouldn't be able to shutdown an entire system, but that should be solved in another way (e.g. resource control for individual processes/threads).
Well, now you are contradicting yourself, IMO. First you assert that zope shouldn't protect against stupidity, then you want to have resource control. Resource control can give a lot of support headaches, and everywhere it is used it causes a lot of mailing list traffic (linux OOM killer is a prominent example). For various reasons the problem to implement something like that in zope would be even more of a headache, I assume, and it's much less needed. Somewhere the line has to be drawn, and I think what is done in zope is quite reasonable, albeit arguable. Anyway, I have no strong feelings one way or the other, just wanted to pass on what I have learned from the same discussion. cheers, oliver
Oliver Bleutgen wrote:
Tue Wennerberg wrote:
Well, now we're getting somewhere. I believe that "guarding against stupidity" is a much more valid point. However, still not valid enough that regular expressions should be banned, since regular expressions would be such a great feature for Zope.
It's not as you couldn't use regexps in zope, it's just not as easy as you like it to be.
In my eyes, a script developer should be trusted to create well-written code. In other words, badly developed scripts cause a badly developed site, which shouldn't surprise anyone. I don't think Zope should (or can) protect against stupidity. In my experience, when non-expert developers create regular expressions, they are always trivial expressions, which don't cause such problems.
Of course a programming error shouldn't be able to shutdown an entire system, but that should be solved in another way (e.g. resource control for individual processes/threads).
Well, now you are contradicting yourself, IMO. First you assert that zope shouldn't protect against stupidity, then you want to have resource control. Resource control can give a lot of support headaches, and everywhere it is used it causes a lot of mailing list traffic (linux OOM killer is a prominent example). For various reasons the problem to implement something like that in zope would be even more of a headache, I assume, and it's much less needed. Somewhere the line has to be drawn, and I think what is done in zope is quite reasonable, albeit arguable. Anyway, I have no strong feelings one way or the other, just wanted to pass on what I have learned from the same discussion.
I appreciate your input, too! I didn't mean to contradict myself :-) What I meant to say was that when choosing between (a) regular expressions working by default, or (b) protecting against rare cases of stupidity, I think (a) should be chosen and I'm surprised it hasn't been. I also think it's bad for Zope that regular expressions have gotten a reputation of being insecure, when they really aren't. On the contrary, the conscientious developer will use them for validating input parameters, thereby increasing security. -- Mvh. Tue Wennerberg Civilingeniør og Freelance Udvikler http://tuewennerberg.dk/ - tue@wennerberg.dk - (+45) 4043 6735
On Sun, Jan 19, 2003 at 09:13:12PM +0100, Tue Wennerberg wrote:
Well, I vaguely remember having participated in a discussion about that also, and I believe I chipped in an example like:
for a in range(0,1000): for b in range(0,1000): for c in range(0,1000):
... you get the picture.
Thus demonstrating that a malicious scripter could always cause a DOS.
Malicious script writer don't need no stinkin' loop: 999999999999999L ** 999999999999999L That will keep Zope completely busy for a nice long while*; due to the way the Global Interpreter Lock works, no other threads will get to run at all as long as this computation is running, because it's only one instruction and the lock can't be released to another thread in the middle of an instruction. Put that in a script, and as soon as someone visits the script, zope is hung until it finishes. Yes, I've tried it. * I don't know how long this will hang for ... I tried it, but got bored of waiting for it to finish so I killed it. Even 99999L ** 99999L takes longer than I care to wait for. -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's ANNOYING RHINOCEROUS CLOWN! (courtesy of isometric.spaceninja.com)
[SKIPPED ALOT] Well, regular expressions are not banned. They are just not allowed in the default setup. I find it more interesting that: from Products.PythonScripts.Utility import allow_module allow_module('re') doesn't give you access to *all* of 're'. AFAIK group is not possible with this approch. \Oliver -- Direktør / Managing Director Oliver Marx TEKK Lyngbyvej 20 DK-2100 København Ø Main: +45 39 15 80 60 Direct: +45 39 15 80 62 http://www.tekk.dk
participants (5)
-
Dylan Reinhardt -
Oliver Bleutgen -
Oliver Marx -
Paul Winkler -
Tue Wennerberg