DTML and REQUEST data changes about to be checked in
Hi folks, I am about to land some big changes in the way DTML deals with data taken from the REQUEST object when accessed implicitly, in both the Zope Trunk and the Zope 2.5 branch. These changes could potentially break existing Zope sites. Without these changes, Zope is somewhat vulnerable to cross-scripting attacks, where a well-crafted URL can cause a Zope server to serve out arbitrary HTML. Because DTML does not automatically html quote any data, and can implicitly get information out of the REQUEST even when it was not the intention of the template author, it is easy to cause REQUEST data to be rendered as HTML on a page. My changes cause the REQUEST to keep track of suspected strings, where suspect is defined as any string with a '<'. These are marked as tainted. Any normal, explicit access to the REQUEST will still give you normal values. However, as soon as a DTML template requests a variable from the general namespace, and this variable is then satisfied from the REQUEST, the value of this variable could potentially be a TaintedString object instead of the original string. When rendering such a value, DTML will automatically HTML quote it if not already done so explicitly. All DTML string operations dealing with TaintedString objects are careful to retain the TaintedString status. I also fixed all exceptions raised in Zope that I could find, where untrusted REQUEST data was used in the exception message; these exceptions now html quote the data. I also made sure that the REQUEST calculated variables URLx and BASEx and such were not shadowed by untrusted form variables of the same name. These changes can break existing sites in the following ways: - If you relied on getting HTML-like data from the REQUEST in DTML and want to render this as HTML, and you got this data implicitly, this data will now be HTML quoted. Note that you were vulnerable to a cross-scripting attack here already. You can retrieve your information from the REQUEST directly (with <dtml-with REQUEST> for example), at your own risk. ;) - HTML quoting will also take place in templates that do not otherwise generate HTML to be sent back to the browser, such as email forms and Z SQL Methods. For Z SQL Methods, <dtml-sqlvar> does not quote TaintedStrings and is otherwise ignorant of them. For emails, use explicit access to REQUEST instead. - If you relied on being able to override URLx or BASEx variables through a form variable, this no longer works. Use explicit access to REQUEST.form instead. - Using the string method .join (''join(items)) cannot handle TaintedString objects. You can use _.string.join instead. - Passing a TaintedString value from a DTML template to other objects such as Python code, External Methods, Python Scripts, etc, may cause them to break because they did not anticipate a TaintedString object. What doesn't break (among others): - Accessing REQUEST data from Python code, Python scripts, or ZPT. Only DocumentTemplate.DT_String derivatives (DTML Document, DTML Method, etc) and DTMLFile objects are affected. - If you already HTML quoted, nothing gets double quoted. - Using the _.string module in DTML retains taints. - Zope 2.6 unicode marshalling (var:ustring:utf8) works with TaintedStrings as well. TaintedString objects try to mimic strings as best as they can, but until we move to python 2.2 definitely and we can inherit from str directly, certain python code will not accept TaintedString objects as substitutes. I found that the normal string module, and the string ''.join module don't accept TaintedString objects for example. Also, using the string interpolation operator % will cause TaintedString objects to be unwrapped. When TaintedString becomes a subclass of str, more operations will unwrap them, such as unicode() and ''.join; or just about any operation that manipulates strings through other ways than string methods. Because of size of the change and the impact on existing DTML code, well release betas of Zope 2.5 and 2.6 soon to facilitate wider testing. For those following CVS, please test the changes rigorously and let me know what you find. -- Martijn Pieters | Software Engineer mailto:mj@zope.com | Zope Corporation http://www.zope.com/ | Creators of Zope http://www.zope.org/ ---------------------------------------------
On Thu, Aug 01, 2002 at 10:46:44AM -0400, Martijn Pieters wrote:
I am about to land some big changes in the way DTML deals with data taken from the REQUEST object when accessed implicitly, in both the Zope Trunk and the Zope 2.5 branch. These changes could potentially break existing Zope sites.
It's in. Let the testing begin! -- Martijn Pieters | Software Engineer mailto:mj@zope.com | Zope Corporation http://www.zope.com/ | Creators of Zope http://www.zope.org/ ---------------------------------------------
On 8/1/02 10:08 AM, "Martijn Pieters" <mj@zope.com> wrote:
On Thu, Aug 01, 2002 at 10:46:44AM -0400, Martijn Pieters wrote:
I am about to land some big changes in the way DTML deals with data taken from the REQUEST object when accessed implicitly, in both the Zope Trunk and the Zope 2.5 branch. These changes could potentially break existing Zope sites.
It's in. Let the testing begin!
Hopefully I'll get a chance to test it with some of our 2.5 sites - I have a small worry that old code on small sites that we don't have much worry about will break if this is put into a 2.5.2 or later release. Could there be a way to disable this "feature" in 2.5 via a z2/environment variable or some other configuration setting, but have it be automatic in 2.6? "Potential code breakage" and "point point release" leave me a little worried about maintaining 2.5 sites. It may not be an issue - I have to digest the changes in more depth that I've had (or currently have) time for, but that's the thought that crossed my mind earlier. -- Jeffrey P Shell www.cuemedia.com
On Thu, Aug 01, 2002 at 10:29:36AM -0600, Jeffrey P Shell wrote:
Hopefully I'll get a chance to test it with some of our 2.5 sites - I have a small worry that old code on small sites that we don't have much worry about will break if this is put into a 2.5.2 or later release. Could there be a way to disable this "feature" in 2.5 via a z2/environment variable or some other configuration setting, but have it be automatic in 2.6? "Potential code breakage" and "point point release" leave me a little worried about maintaining 2.5 sites.
It may not be an issue - I have to digest the changes in more depth that I've had (or currently have) time for, but that's the thought that crossed my mind earlier.
From a technical standpoint I can indeed add a switch that would disable the occurence of tainted strings, yes. I'll discuss this with Brian, it shouldn't be hard to add. But note that breakage only occurs when REQUEST data actually contains possibly dangerous markup, and your site was vulnerable in those areas that now break. Disabeling the tainting will leave you vulnerable. -- Martijn Pieters | Software Engineer mailto:mj@zope.com | Zope Corporation http://www.zope.com/ | Creators of Zope http://www.zope.org/ ---------------------------------------------
On Thu, Aug 01, 2002 at 12:34:30PM -0400, Martijn Pieters wrote:
On Thu, Aug 01, 2002 at 10:29:36AM -0600, Jeffrey P Shell wrote:
Hopefully I'll get a chance to test it with some of our 2.5 sites - I have a small worry that old code on small sites that we don't have much worry about will break if this is put into a 2.5.2 or later release. Could there be a way to disable this "feature" in 2.5 via a z2/environment variable or some other configuration setting, but have it be automatic in 2.6? "Potential code breakage" and "point point release" leave me a little worried about maintaining 2.5 sites.
It may not be an issue - I have to digest the changes in more depth that I've had (or currently have) time for, but that's the thought that crossed my mind earlier.
From a technical standpoint I can indeed add a switch that would disable the occurence of tainted strings, yes. I'll discuss this with Brian, it shouldn't be hard to add.
But note that breakage only occurs when REQUEST data actually contains possibly dangerous markup, and your site was vulnerable in those areas that now break. Disabeling the tainting will leave you vulnerable.
Just checked into CVS for both 2.5 and 2.6; setting ZOPE_DTML_REQUEST_AUTOQUOTE to one of 'no', '0', or 'disabled' will disable the new tainting of strings and thus disable autoquoting. -- Martijn Pieters | Software Engineer mailto:mj@zope.com | Zope Corporation http://www.zope.com/ | Creators of Zope http://www.zope.org/ ---------------------------------------------
Likewise Im trying to digest all that and Im a little suprised. More magic in DTML? Not something I'd vote for normally. Im a little confused why this is suddenly an issue, yeah so we pull a string out of the REQUEST and thanks to DTML stack we may not know where it came from. Well thats always been there. And yeah the string may contain nasty HTML. Again that's always been there. In the past (and I cant find posts to show it) the party line was Zope is an application server and its up to the person developing the application to worry about it. Thats why ChrisW wrote stripogram and I use it in quite a few apps. One other question? Why does it matter that the string is implicitly called, why dont you taint explicitly called to? It makes me think of Perl where taint mode taints anything coming from the user? This still doesnt solve the party line and means I would like to suggest again (and this time I have the time to work on it) that we add something like stripogram or similar to the core, so that is easy for an application developer to have access to strip html and other functions from products, DTML, Python Scripts etc to easily alter, manage and make HTML safer. -- Andy McKay @gmweb Consulting http://www.agmweb.ca
On Fri, Aug 02, 2002 at 08:55:13AM -0700, Andy McKay wrote:
Likewise Im trying to digest all that and Im a little suprised. More magic in DTML? Not something I'd vote for normally.
Im a little confused why this is suddenly an issue, yeah so we pull a string out of the REQUEST and thanks to DTML stack we may not know where it came from. Well thats always been there. And yeah the string may contain nasty HTML. Again that's always been there. In the past (and I cant find posts to show it) the party line was Zope is an application server and its up to the person developing the application to worry about it. Thats why ChrisW wrote stripogram and I use it in quite a few apps.
Yup. And that is still the case. However, the combination of implict REQUEST form interpolation and no HTML quoting turns out to especially dangerous, because of those situations where you *want* no HTML quoting for optional information that normally should *not* come from the REQUEST. An example is the Zope help system; there are API help pages that have optional information, which when present is already HTML. But when not present in the object hierarchy, but it *is* available in the REQUEST, the REQUEST data is used instead. The way standard_error_message deals with exceptions is another such a situation. The DTML author didn't expect the particular template slot to be filled with REQUEST data, the slot is optional, and the author has no way of preventing REQUEST data from being used. The solution we choose fixes that problem, for all existing DTML as well as future DTML. Note that ZPT does not have this problem, as it quotes by default and doesn't use implict namespaces.
One other question? Why does it matter that the string is implicitly called, why dont you taint explicitly called to? It makes me think of Perl where taint mode taints anything coming from the user?
Because, as explained above, its the implicit case that is dangerous. In the explicit case you are supposed to know you are working with unsafe data and thus the old rules apply. If we explicitly quoted, we hurt everyone that either did the right thing from the start and/or already knows they are playing with fire.
This still doesnt solve the party line and means I would like to suggest again (and this time I have the time to work on it) that we add something like stripogram or similar to the core, so that is easy for an application developer to have access to strip html and other functions from products, DTML, Python Scripts etc to easily alter, manage and make HTML safer.
The CMF now includes a basic HTML stripper. In future iterations, Tres Seaver expects this to evolve into a CMF Tool that is more generaly configurable and useable. -- Martijn Pieters | Software Engineer mailto:mj@zope.com | Zope Corporation http://www.zope.com/ | Creators of Zope http://www.zope.org/ ---------------------------------------------
The way I see it is this: ZPT has (amongst others) the very nice property that it's trivial to audit ZPT pages for non-quoted string output: just grep for 'structure'. OTOH in DTML there's no way to do that, the default is unquoted output. &dtml-stuff; is a good step, but there's way too much code out there that doesn't take proper security precautions and is vulnerable to cross-site scripting attacks. The fact that namespaces are implicit and make it possible to get stuff from where we don't expect them doesn't help. We should always keep in mind the security debacle that PHP has become, because it didn't take proper precautions for quoting (reports on Bugtraq of XSS attacks in some PHP application practically every week). There's magic_quotes_gpc or magic_quotes_runtime that automatically quotes their equivalent of REQUEST or even any output from outside applications, but it's a PITA because it does too much and is often inconvenient, so a lot of users turn it off or forget to turn it on. And it's too much magic. The solution Martijn implemented has a number of nice properties, for instance protecting those that do <dtml-var foo>, and will only leave open the applications that really didn't take the most basic security precautions (unfortunately I'm sure there are some out there, doing stuff like <a href="<dtml-var "REQUEST.foo">"><dtml-var "REQUEST.footitle"></a>). So in a code audit all <dtml-var [expr=]"xxx"> should be suspect, and &dtml-.xxx; too. Florent -- Florent Guillaume, Nuxeo (Paris, France) +33 1 40 33 79 87 http://nuxeo.com mailto:fg@nuxeo.com
On Thursday 01 Aug 2002 3:46 pm, Martijn Pieters wrote: Eeek - a week ago. sorry for the late reply.
I am about to land some big changes in the way DTML deals with data taken from the REQUEST object when accessed implicitly, in both the Zope Trunk and the Zope 2.5 branch.
In my opinion this change is completely unacceptable at this late stage of the release cycle. As you said:
These changes could potentially break existing Zope sites.
The existing behavior might be flawed, but it is a flaw we have all lived with for a long time. In my opinion this needs: 1. To be deferred until the 2.7 cycle. 2. A detailed fishbowl proposal. ps. I think your idea is a good one.
On 8/8/02 1:19 PM, "Toby Dickenson" <tdickenson@geminidataloggers.com> wrote:
On Thursday 01 Aug 2002 3:46 pm, Martijn Pieters wrote:
Eeek - a week ago. sorry for the late reply.
I am about to land some big changes in the way DTML deals with data taken from the REQUEST object when accessed implicitly, in both the Zope Trunk and the Zope 2.5 branch.
In my opinion this change is completely unacceptable at this late stage of the release cycle. As you said:
These changes could potentially break existing Zope sites.
The existing behavior might be flawed, but it is a flaw we have all lived with for a long time. In my opinion this needs:
1. To be deferred until the 2.7 cycle.
2. A detailed fishbowl proposal.
Speaking of all this - what's up with 2.6 and 2.7? 2.6a1 is creaking towards to its two month birthday, and I seem to hear more about 2.7 developments than 2.6 ones lately. Is there a roadmap / schedule for the two releases that's up to date? -- Jeffrey P Shell www.cuemedia.com
On Thu, Aug 08, 2002 at 08:19:12PM +0100, Toby Dickenson wrote:
I am about to land some big changes in the way DTML deals with data taken from the REQUEST object when accessed implicitly, in both the Zope Trunk and the Zope 2.5 branch.
In my opinion this change is completely unacceptable at this late stage of the release cycle. As you said:
These changes could potentially break existing Zope sites.
The existing behavior might be flawed, but it is a flaw we have all lived with for a long time. In my opinion this needs:
1. To be deferred until the 2.7 cycle.
2. A detailed fishbowl proposal.
Note that the problems fixed are potential security problems. Although we cannot fix every site out there for sure, the fixes certainly dramatically reduce the risks. The risk for breakage is very small really, and breakage will generally only occur when someone is trying to exploit the weakness, not in normal operation of the site. I'll leave any decisions on wether or not this stays in the current release cycles or moves to 2.7 to Jim Fulton. He is unfortunately on cvacation until next week. -- Martijn Pieters | Software Engineer mailto:mj@zope.com | Zope Corporation http://www.zope.com/ | Creators of Zope http://www.zope.org/ ---------------------------------------------
On Thursday 08 Aug 2002 9:29 pm, Martijn Pieters wrote:
On Thu, Aug 08, 2002 at 08:19:12PM +0100, Toby Dickenson wrote:
I am about to land some big changes in the way DTML deals with data taken from the REQUEST object when accessed implicitly, in both the Zope Trunk and the Zope 2.5 branch.
In my opinion this change is completely unacceptable at this late stage of
the release cycle. As you said:
These changes could potentially break existing Zope sites.
The existing behavior might be flawed, but it is a flaw we have all lived with for a long time. In my opinion this needs:
1. To be deferred until the 2.7 cycle.
2. A detailed fishbowl proposal.
Note that the problems fixed are potential security problems. Although we cannot fix every site out there for sure, the fixes certainly dramatically reduce the risks.
Im not going to argue that this feature is bad - because I dont believe that to be true. I suspect the feature is not exactly quite right - but those issues can easily be resolved over a full release cycle.
The risk for breakage is very small really
Your choice of '<' and html_quote suggests that my dtml code which generates javascript and vbscript carries a higher risk than dtml which generates html.
, and breakage will generally only occur when someone is trying to exploit the weakness, not in normal operation of the site.
The fact that your change uses html_quote to 'fix' the problem rather than sounding 'hacker alert' alarm bells suggests to me that you dont really believe that ;-)
I'll leave any decisions on wether or not this stays in the current release cycles or moves to 2.7 to Jim Fulton. He is unfortunately on cvacation until next week.
On Fri, Aug 09, 2002 at 09:56:45AM +0100, Toby Dickenson wrote:
The risk for breakage is very small really
Your choice of '<' and html_quote suggests that my dtml code which generates javascript and vbscript carries a higher risk than dtml which generates html.
Only if you generated that script using data from the REQUEST, implicitly. Which was bad in the first place.
, and breakage will generally only occur when someone is trying to exploit the weakness, not in normal operation of the site.
The fact that your change uses html_quote to 'fix' the problem rather than sounding 'hacker alert' alarm bells suggests to me that you dont really believe that ;-)
Again, the wide scope of DTML use would make such bells warble prematurely all too often. The normal, recommended fix for the general weakness is to always use HTML quote. -- Martijn Pieters | Software Engineer mailto:mj@zope.com | Zope Corporation http://www.zope.com/ | Creators of Zope http://www.zope.org/ ---------------------------------------------
On Friday 09 Aug 2002 3:12 pm, Martijn Pieters wrote:
On Fri, Aug 09, 2002 at 09:56:45AM +0100, Toby Dickenson wrote:
The risk for breakage is very small really
Your choice of '<' and html_quote suggests that my dtml code which generates javascript and vbscript carries a higher risk than dtml which generates html.
Only if you generated that script using data from the REQUEST, implicitly.
Yes
Which was bad in the first place.
I agree it is true in most cases, but not all. Have you analysed how many applications will be broken by this? how they can detect the breakage? I certainly will not have time to assess the implications on my applications before the scheduled release of 2.6.
, and breakage will generally only occur when someone is trying to exploit the weakness, not in normal operation of the site.
The fact that your change uses html_quote to 'fix' the problem rather than sounding 'hacker alert' alarm bells suggests to me that you dont really believe that ;-)
Again, the wide scope of DTML use would make such bells warble prematurely all too often.
'all too often' also contradicts your statements that this will not happen in normal operation of the site, and that the risk of breakage is 'very small'. Like I said before, this is probably a good feature. If it was available as a patch then I would probably use it on a number of my sites, and would recommend it to others. I would be very happy see it (or something like it) in 2.7. But not 2.6.
On Fri, 2002-08-09 at 10:43, Toby Dickenson wrote:
On Friday 09 Aug 2002 3:12 pm, Martijn Pieters wrote:
On Fri, Aug 09, 2002 at 09:56:45AM +0100, Toby Dickenson wrote:
The risk for breakage is very small really
Your choice of '<' and html_quote suggests that my dtml code which generates javascript and vbscript carries a higher risk than dtml which generates html.
Only if you generated that script using data from the REQUEST, implicitly.
Yes
Which was bad in the first place.
I agree it is true in most cases, but not all. Have you analysed how many applications will be broken by this? how they can detect the breakage? I certainly will not have time to assess the implications on my applications before the scheduled release of 2.6.
, and breakage will generally only occur when someone is trying to exploit the weakness, not in normal operation of the site.
The fact that your change uses html_quote to 'fix' the problem rather than sounding 'hacker alert' alarm bells suggests to me that you dont really believe that ;-)
Again, the wide scope of DTML use would make such bells warble prematurely all too often.
'all too often' also contradicts your statements that this will not happen in normal operation of the site, and that the risk of breakage is 'very small'.
Like I said before, this is probably a good feature. If it was available as a patch then I would probably use it on a number of my sites, and would recommend it to others. I would be very happy see it (or something like it) in 2.7.
But not 2.6.
Martijn did add a knob to turn the feature off, via a new environment variable. With a security vulnerability, we have to come up with some kind of balance between the need to propagate the fix as quickly as possible and the need (as you point out) not to disrupt production sites unduly. I don't believe we can afford to wait a whole other release cycle for this fix; Brian, Jim, and Martijn deemed the fix too pervasive to be bundled as a hotfix, which offers us little choice except to included it in current releases. Whithout the fix, virtually every Zope site in the world is vulnerable to URL-based cross-site scripting exploits. For instance, any URL which contains invalid form variable marshalling can generate an error page which includes the erroneous value, unquoted. E.g.: <URL:http://somezopesite.com/looks/like/legitimate?foo:int=%3Cscript%3Ealert('Owned')%3C/script%3E> Tres. -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com
Tres Seaver wrote:
Martijn did add a knob to turn the feature off, via a new environment variable. With a security vulnerability, we have to come up with some kind of balance between the need to propagate the fix as quickly as possible and the need (as you point out) not to disrupt production sites unduly. I don't believe we can afford to wait a whole other release cycle for this fix; Brian, Jim, and Martijn deemed the fix too pervasive to be bundled as a hotfix, which offers us little choice except to included it in current releases.
Whithout the fix, virtually every Zope site in the world is vulnerable to URL-based cross-site scripting exploits. For instance, any URL which contains invalid form variable marshalling can generate an error page which includes the erroneous value, unquoted. E.g.:
<URL:http://somezopesite.com/looks/like/legitimate?foo:int=%3Cscript%3Ealert('Owned')%3C/script%3E>
Yeah, but why not fixing that instead? I for one concur that it is a good thing to care for such stuff, but the way it is fixed might really lead to a lot of confusion, because it again introduces some magic which happens behind the users back. As far as I understand the fix, I can imagine some use cases which might break without having been in danger at all. Someone please correct me if I'm wrong. 1. Sending mails with dtml-sendmail If you send plaintext mails, suddendly you get html_quoted stuff which you never wanted. Having to debug this without knowing about the new "feature" will get people up in arms. 2. <textarea ...><dtml-var some_variable_contained_in_request></textarea> 3. Any object which uses dtml to render content from the REQUEST not meant for being displayed in a browser. Some people might use zope for things which have nothing to do with html, so they didn't care about cross-site scripting in the first place. cheers, oliver
Tres Seaver wrote:
Whithout the fix, virtually every Zope site in the world is vulnerable to URL-based cross-site scripting exploits. For instance, any URL which contains invalid form variable marshalling can generate an error page which includes the erroneous value, unquoted. E.g.:
<URL:http://somezopesite.com/looks/like/legitimate?foo:int=%3Cscript%3Ealert('Owned')%3C/script%3E>
I think an URL with an inconspicuous mispelling of the domain name is a far greater vulnerability than cross-site-scripting. Consider: http://barnesandnohle.com/freebooks.html An attacker could set up that misleading domain name then spam people to order free books, requiring credit card info for some "book-of-the-month" trick. Some simple scraping of the true site would keep most people from ever thinking there was a problem. In fact, you don't even have to misspell it: http://barnes-andnoble.com/freebooks.html This affects the entire web and every piece of software involved in it. I just want to keep the security worries in check. Let me ramble for a bit... We've released a lot of hotfixes, but *none* of the vulnerabilities could give an attacker root access, and none of them could give console access to anonymous users AFAIK. All of the vulnerabilities violated Zope's security policy, but Zope's security policy is constrained by system security and other safeguards. People outside the Zope community don't know that, so a lot have labeled Zope as too insecure to use. The reality is that we've never even had an exploitable buffer overrun. :-) We should avoid sending the wrong message by making a hotfix for every little thing. Shane
[Snip]
I just want to keep the security worries in check. Let me ramble for a bit... We've released a lot of hotfixes, but *none* of the vulnerabilities could give an attacker root access, and none of them could give console access to anonymous users AFAIK. All of the vulnerabilities violated Zope's security policy, but Zope's security policy is constrained by system security and other safeguards. People outside the Zope community don't know that, so a lot have labeled Zope as too insecure to use. The reality is that we've never even had an exploitable buffer overrun. :-) We should avoid sending the wrong message by making a hotfix for every little thing.
Shane
I'd like to second this. It was one of the contibuting factors in the decision of my former employers to opt for spectra instead of a Zope solution (That already existed!!). I am sure there are other cases of this too... If someone finds a buffer overrun, fix it by all means, but other issues may be better left for minor version releases, where they can be buried in the changelog. Just my £0.02 Adrian... -- Adrian Hungate EMail: adrian@haqa.co.uk Web: http://www.haqa.co.uk
Adrian Hungate writes:
....
We should avoid sending the wrong message by making a hotfix for every little thing.
Shane
I'd like to second this. It was one of the contibuting factors in the decision of my former employers to opt for spectra instead of a Zope solution (That already existed!!). I, in contrary, appreciate the openess and fast response with respect to security problems.
I do not install most hotfixes because the vulnerabilities do not affect our sites but it is a good feeling that there are fast fixes when this would be once the case. Dieter
Dieter Maurer wrote:
Adrian Hungate writes:
....
We should avoid sending the wrong message by making a hotfix for every little thing.
Shane
I'd like to second this. It was one of the contibuting factors in the decision of my former employers to opt for spectra instead of a Zope solution (That already existed!!). I, in contrary, appreciate the openess and fast response with respect to security problems.
I do not install most hotfixes because the vulnerabilities do not affect our sites but it is a good feeling that there are fast fixes when this would be once the case.
In some way we need to make it clear that most hotfixes don't matter for most sites. A lot of hotfixes ensured that users who could write DTML couldn't get extra privileges. They really only mattered for sites like zope.org, where anyone with an email address is allowed to write code that will be executed directly on the server. But: 1) most Zope sites give a high level of trust to DTML authors anyway. There was no way to exploit most of the security holes without the ability to write DTML that runs on the server. 2) even a Zope administrator is still quite limited. In a standard setup, a Zope admin can't read/write arbitrary files or execute scripts. 3) Zope doesn't run as root. Even if someone found a way to get console access through a Zope admin account, they would have to exploit some other security hole to get root access. We need to make it clear that there are several layers of security, and only a single layer has ever had a problem AFAIK. Shane
I'd like to second this. It was one of the contibuting factors in the decision of my former employers to opt for spectra instead of a Zope solution (That already existed!!). I, in contrary, appreciate the openess and fast response with respect to security problems.
I do not install most hotfixes because the vulnerabilities do not affect our sites but it is a good feeling that there are fast fixes when this would be once the case.
In some way we need to make it clear that most hotfixes don't matter for most sites. A lot of hotfixes ensured that users who could write DTML couldn't get extra privileges. They really only mattered for sites like zope.org, where anyone with an email address is allowed to write code that will be executed directly on the server. But:
You are exactly right... but... The problem is not one of clarity of labling, it's one of targetting: The people that actually make this level of decision (i.e. board level execs) are not "techies", and are just not interested in _why_ the fix is needed, or in _what_ technical problem it fixes, but that ZC (visibly) releases 'n' fixes per month for Zope, while M$ (visibly) releases less than that number per year for IIS/ASP - Therefore, Zope must be the less stable/reliable product etc? The logic is flawed, we all know that, but who is volunteering to visit every companies senior execs worldwide and spend the time to make them care enough? The hotfixes, and new releases need to be "marketted" (I use that word loosely) quite differently, new releases are "A Good Thing(tm)", while the fixes need to be "under the hood" where the execs won't be bothered by them, but the techies can find them when they need them. I appologise, in advance, for the sweeping generalization that all execs are like Dilbert's pointy-haired boss, but some really are! Adrian... -- Adrian Hungate EMail: adrian@haqa.co.uk Web: http://www.haqa.co.uk
On Friday 09 Aug 2002 4:33 pm, Tres Seaver wrote:
Whithout the fix, virtually every Zope site in the world is vulnerable to URL-based cross-site scripting exploits. For instance, any URL which contains invalid form variable marshalling can generate an error page which includes the erroneous value, unquoted. E.g.:
<URL:http://somezopesite.com/looks/like/legitimate?foo:int=%3Cscript%3Ealer t('Owned')%3C/script%3E>
Do you plan to fix this bug? Or, with the autoquoting changes, is this to be reclassified as 'not a bug'?
On Mon, Aug 12, 2002 at 03:51:24PM +0100, Toby Dickenson wrote:
On Friday 09 Aug 2002 4:33 pm, Tres Seaver wrote:
Whithout the fix, virtually every Zope site in the world is vulnerable to URL-based cross-site scripting exploits. For instance, any URL which contains invalid form variable marshalling can generate an error page which includes the erroneous value, unquoted. E.g.:
<URL:http://somezopesite.com/looks/like/legitimate?foo:int=%3Cscript%3Ealer t('Owned')%3C/script%3E>
Do you plan to fix this bug?
Or, with the autoquoting changes, is this to be reclassified as 'not a bug'?
Together with the autoquoting changes, I tightened Exception messages; data from REQUEST is quoted where I could reasonably suspect REQUEST data was used. -- Martijn Pieters | Software Engineer mailto:mj@zope.com | Zope Corporation http://www.zope.com/ | Creators of Zope http://www.zope.org/ ---------------------------------------------
On 8/9/02 8:43 AM, "Toby Dickenson" <tdickenson@geminidataloggers.com> wrote:
I agree it is true in most cases, but not all. Have you analysed how many applications will be broken by this? how they can detect the breakage? I certainly will not have time to assess the implications on my applications before the scheduled release of 2.6.
This is why I raised the flag of "can there be a way to disable it?", and Martijn put a fix in: - <dtml-var name> and &dtml.-name; will now automatically HTML-quote unsafe data taken implictly from the REQUEST object. Data taken explicitly from the REQUEST object is not affected, as well as any other data not originating from REQUEST. This can be disabled (at your own risk!) by setting the environment variable ZOPE_DTML_REQUEST_AUTOQUOTE to one of 'no', '0', or 'disabled'. I have the same concerns you do, but I figure that if any problems are found during normal execution of any Zope release this is attached to that I don't have time to investigate a fix for myself, I can add this environment variable (which normally I am not fond of doing), restart, and make a note "investigate fixing site blablabla". Is there any reason why this solution wouldn't work for you?
Like I said before, this is probably a good feature. If it was available as a patch then I would probably use it on a number of my sites, and would recommend it to others. I would be very happy see it (or something like it) in 2.7.
But not 2.6.
Oh, 2.6 will never happen anyways ;) (seriously folks - what's the plan?). Since there's no current release plan for 2.6, it's hard to plan future deployments around it anyways. But if you have any sites you plan to move to 2.6, you should test this Autoquote change aggressively during the alpha/beta cycle. Since the ZOPE_DTML_REQUEST_AUTOQUOTE change has been put in, I've reserved future judgments until I get a chance to actually do some testing. I know that if I do run into any issues in the future that I don't have time to deal with, I can just flip that switch off. -- Jeffrey P Shell www.cuemedia.com
Jeffrey P Shell wrote:
On 8/9/02 8:43 AM, "Toby Dickenson" <tdickenson@geminidataloggers.com> wrote:
...
Like I said before, this is probably a good feature. If it was available as a patch then I would probably use it on a number of my sites, and would recommend it to others. I would be very happy see it (or something like it) in 2.7.
But not 2.6.
Oh, 2.6 will never happen anyways ;) (seriously folks - what's the plan?).
The plan is to release 2.6 as soon as we can. We're really busy with a bunch of customer work and haven't had as much time to work on this as we'd like. It's likely that 2.7 will come out at around the same time as 2.6. See http://dev.zope.org/Wikis/DevSite/Projects/SupportPython22/VisionStatement. WRT to this change, now that I'm back from vacation, I want to talk to Brian about it. ;) Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (888) 344-4332 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
Jim Fulton wrote:
The plan is to release 2.6 as soon as we can. We're really busy with a bunch of customer work and haven't had as much time to work on this as we'd like.
Well actually this is more comforting than if you had a lot of time on your hands ;-) regards Max M -- "Sorry I would Really Like To Help More On This Project, But Am To Busy Doing Paid Work On A Tight Deadline" Max M
Like I said before, this is probably a good feature. If it was available as a patch then I would probably use it on a number of my sites, and would recommend it to others. I would be very happy see it (or something like it) in 2.7.
But not 2.6.
Then Jim wrote: WRT to this change, now that I'm back from vacation, I want to talk to Brian about it. ;)
Hear ye, hear ye :^) Zope 2.6 is a second-dot release, meaning that it is expected that there will be new features and that it is possible (though we always try to avoid it) that some things can break in the name of progress. (See http://dev.zope.org/CVS/ZopeReleasePolicy for more details). Zope 2.5.x will be a third-dot release, intended to be bug-fix only (and thus not allowed to break things). So here's what we'll do. Zope 2.6 will include the string tainting changes, enabled by default. The tainting can be turned off by providing an environment variable. The next Zope 2.5.x release will contain the tainting code, but it will be *disabled* by default. If you are worried about the issues it addresses, you will be able to enable it explicitly using an environment variable (without having to upgrade to 2.6). 2.7 and later releases will behave as 2.6. Brian Lloyd brian@zope.com V.P. Engineering 540.361.1716 Zope Corporation http://www.zope.com
On Wed, Aug 14, 2002 at 04:25:09PM -0400, Brian Lloyd wrote:
So here's what we'll do. Zope 2.6 will include the string tainting changes, enabled by default. The tainting can be turned off by providing an environment variable.
The next Zope 2.5.x release will contain the tainting code, but it will be *disabled* by default. If you are worried about the issues it addresses, you will be able to enable it explicitly using an environment variable (without having to upgrade to 2.6).
I checked in the changes for 2.5; auto quoating now has to be enabled with an environment variable. Higly recommended! -- Martijn Pieters | Software Engineer mailto:mj@zope.com | Zope Corporation http://www.zope.com/ | Creators of Zope http://www.zope.org/ ---------------------------------------------
participants (13)
-
Adrian Hungate -
Andy McKay -
Brian Lloyd -
Dieter Maurer -
Florent Guillaume -
Jeffrey P Shell -
Jim Fulton -
Martijn Pieters -
Max M -
Oliver Bleutgen -
Shane Hathaway -
Toby Dickenson -
Tres Seaver