I started a branch for doing W3C HTML validation on responses to zope.testbrowser requests: svn://svn.zope.org/repos/main/zope.testbrowser/branches/rossp-validator The idea is to be able to flip a switch and run all my functional zope.testbrowser tests and see validation failures as test failures. The current implementation in that branch just uses a simple environment variable since I wasn't sure what the best way was to support flipping a switch like that. Note that it's possible to use a local installation of the W3C validator (such as is available on Debian based systems in the w3c-markup-validator package) so that the validation doesn't actually slow tests down all that much. I have a couple of questions I'd like to hear any thoughts on. What's the best way to support "flipping the switch" in a global sense so that during the inner testing loop extra time isn't being wasted on repeated validations? Does this belong in zope.testbrowser or in the underlying mechanize package? Other than that, what's the path forward to merging? Ross
Ross Patterson <me@rpatterson.net> writes:
I started a branch for doing W3C HTML validation on responses to zope.testbrowser requests:
svn://svn.zope.org/repos/main/zope.testbrowser/branches/rossp-validator
The idea is to be able to flip a switch and run all my functional zope.testbrowser tests and see validation failures as test failures. The current implementation in that branch just uses a simple environment variable since I wasn't sure what the best way was to support flipping a switch like that. Note that it's possible to use a local installation of the W3C validator (such as is available on Debian based systems in the w3c-markup-validator package) so that the validation doesn't actually slow tests down all that much.
I have a couple of questions I'd like to hear any thoughts on.
What's the best way to support "flipping the switch" in a global sense so that during the inner testing loop extra time isn't being wasted on repeated validations?
Does this belong in zope.testbrowser or in the underlying mechanize package?
Another option would be to integrate this with the publisher itself. For example, an option could be provided for validating the HTML to be returned by the publisher whenever it's running in validating mode, similar to debug mode. It seems like this could make the validating service more accessible to less technical users. I'm not really sure where this belongs. Ross
Other than that, what's the path forward to merging?
Ross
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Hi, (removed plone.devel cross-post) On 02/20/2010 01:53 PM, Ross Patterson wrote:
I started a branch for doing W3C HTML validation on responses to zope.testbrowser requests:
svn://svn.zope.org/repos/main/zope.testbrowser/branches/rossp-validator
The idea is to be able to flip a switch and run all my functional zope.testbrowser tests and see validation failures as test failures. The current implementation in that branch just uses a simple environment variable since I wasn't sure what the best way was to support flipping a switch like that. Note that it's possible to use a local installation of the W3C validator (such as is available on Debian based systems in the w3c-markup-validator package) so that the validation doesn't actually slow tests down all that much.
I have a couple of questions I'd like to hear any thoughts on.
What's the best way to support "flipping the switch" in a global sense so that during the inner testing loop extra time isn't being wasted on repeated validations?
I would not look at environment variables within the testbrowser code at all but stick to Python's mechanics to store "global" or local data. The test runner could grow an option then which can influence that switch beforehand. (E.g. via a feature) I think that's one of the valid use cases of "arbitrary" plugins for the test runner. Christian -- Christian Theune · ct@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development
On Sat, Feb 20, 2010 at 7:53 AM, Ross Patterson <me@rpatterson.net> wrote:
I started a branch for doing W3C HTML validation on responses to zope.testbrowser requests:
svn://svn.zope.org/repos/main/zope.testbrowser/branches/rossp-validator
The idea is to be able to flip a switch and run all my functional zope.testbrowser tests and see validation failures as test failures.
I'm not keen on this idea. Validation checking isn't hard to do with testbrowser as-is, and doing it on every page load is overkill. Add to that the global nature of the required switch and this really doesn't seem like the right approach. I suggest you create a package that subclasses testbrowser to provide the behavior you want. Perhaps if it is widely adopted and the approach proves itself, it can be rolled in.
Does this belong in zope.testbrowser or in the underlying mechanize package?
It appears to me to belong higher in the stack, not lower. I do like the local validator approach. How will you manage that dependency? -- Benji York
On Tue, Feb 23, 2010 at 08:57:09AM -0500, Benji York wrote:
On Sat, Feb 20, 2010 at 7:53 AM, Ross Patterson <me@rpatterson.net> wrote:
I started a branch for doing W3C HTML validation on responses to zope.testbrowser requests:
svn://svn.zope.org/repos/main/zope.testbrowser/branches/rossp-validator
The idea is to be able to flip a switch and run all my functional zope.testbrowser tests and see validation failures as test failures.
I'm not keen on this idea. Validation checking isn't hard to do with testbrowser as-is, and doing it on every page load is overkill. Add to that the global nature of the required switch and this really doesn't seem like the right approach.
I'd like to chime in and suggest using WSGI to hook up zope.testbrowser and zope.app.testing.functional (or a better-designed replacement for the latter). Then you could plug in W3CValidatingMiddleware (if one doesn't exist, it should be trivial to write it) between them. Marius Gedminas -- http://pov.lt/ -- Zope 3 consulting and development
On Tue, Feb 23, 2010 at 9:57 AM, Marius Gedminas <marius@gedmin.as> wrote:
I'd like to chime in and suggest using WSGI to hook up zope.testbrowser and zope.app.testing.functional (or a better-designed replacement for the latter). Then you could plug in W3CValidatingMiddleware (if one doesn't exist, it should be trivial to write it) between them.
+1 The ability to plug in testing code into the WSGI pipeline the same way we plug in production code into a WSGI server pipeline sounds cool. -- Benji York
On Tue, Feb 23, 2010 at 04:57:02PM +0200, Marius Gedminas wrote:
On Tue, Feb 23, 2010 at 08:57:09AM -0500, Benji York wrote:
On Sat, Feb 20, 2010 at 7:53 AM, Ross Patterson <me@rpatterson.net> wrote:
I started a branch for doing W3C HTML validation on responses to zope.testbrowser requests:
svn://svn.zope.org/repos/main/zope.testbrowser/branches/rossp-validator
The idea is to be able to flip a switch and run all my functional zope.testbrowser tests and see validation failures as test failures.
I'm not keen on this idea. Validation checking isn't hard to do with testbrowser as-is, and doing it on every page load is overkill. Add to that the global nature of the required switch and this really doesn't seem like the right approach.
I'd like to chime in and suggest using WSGI to hook up zope.testbrowser and zope.app.testing.functional (or a better-designed replacement for the latter). Then you could plug in W3CValidatingMiddleware (if one doesn't exist, it should be trivial to write it) between them.
We use wsgi_intercept to get zope.testbrowser to listen to the other end of a WSGI pipe during testing http://pypi.python.org/pypi/wsgi_intercept The testbrowser integration we have is far from perfect but was enough for is to stop using zope.app.testing.functional http://svn.zope.org/van.testing/trunk/van/testing/layer.py?rev=105324&view=m... It would be pretty simple to have a make_application call that wrapped the application in validating middleware depending on an environment variable. -- Brian Sutherland
Brian Sutherland <brian@vanguardistas.net> writes:
On Tue, Feb 23, 2010 at 04:57:02PM +0200, Marius Gedminas wrote:
On Tue, Feb 23, 2010 at 08:57:09AM -0500, Benji York wrote:
On Sat, Feb 20, 2010 at 7:53 AM, Ross Patterson <me@rpatterson.net> wrote:
I started a branch for doing W3C HTML validation on responses to zope.testbrowser requests:
svn://svn.zope.org/repos/main/zope.testbrowser/branches/rossp-validator
The idea is to be able to flip a switch and run all my functional zope.testbrowser tests and see validation failures as test failures.
I'm not keen on this idea. Validation checking isn't hard to do with testbrowser as-is, and doing it on every page load is overkill. Add to that the global nature of the required switch and this really doesn't seem like the right approach.
I'd like to chime in and suggest using WSGI to hook up zope.testbrowser and zope.app.testing.functional (or a better-designed replacement for the latter). Then you could plug in W3CValidatingMiddleware (if one doesn't exist, it should be trivial to write it) between them.
We use wsgi_intercept to get zope.testbrowser to listen to the other end of a WSGI pipe during testing
Yeah, I've looked at wsgi_intercept before. Good stuff.
The testbrowser integration we have is far from perfect but was enough for is to stop using zope.app.testing.functional
http://svn.zope.org/van.testing/trunk/van/testing/layer.py?rev=105324&view=m...
Thanks, I'll check it out.
It would be pretty simple to have a make_application call that wrapped the application in validating middleware depending on an environment variable.
Yeah, the more I've thought about it, the more I've been thinking the right place to do this is in the application's publisher somehow, such as in a WSGI pipeline. This would make the validation more widely usable, specifically to non-developers. It would me more of a kind of debug mode. Lo and behold, it's been done: http://workaround.org/node/266 Appending the validation errors to the end of the page won't work for using existing functional tests as a validation testbed because no error is raised on validation failures and many functional tests ignore the parts of the returned page they're not interested in. As such, the tests won't fail on validation errors. I can't imagine it would be too hard to change this behavior. I'll have to try adding WSGI to one of my typical Zope 2.10, Plone 3 add-on testing buildouts to see how much of a burden this places. More later, Ross
Benji York <benji@zope.com> writes:
On Sat, Feb 20, 2010 at 7:53 AM, Ross Patterson <me@rpatterson.net> wrote:
I started a branch for doing W3C HTML validation on responses to zope.testbrowser requests:
svn://svn.zope.org/repos/main/zope.testbrowser/branches/rossp-validator
The idea is to be able to flip a switch and run all my functional zope.testbrowser tests and see validation failures as test failures.
I'm not keen on this idea. Validation checking isn't hard to do with testbrowser as-is, and doing it on every page load is overkill. Add to that the global nature of the required switch and this really doesn't seem like the right approach.
I suggest you create a package that subclasses testbrowser to provide the behavior you want. Perhaps if it is widely adopted and the approach proves itself, it can be rolled in.
This means I wouldn't be able to run *existing* functional tests as a validation check. Since many packages include functional tests that *already* exercise the HTML generation in the application, I want to use them to validate the HTML. You're right that validation checking isn't hard to do with testbrowser as-is if you're trying to write tests *dedicated* to having coverage for your HTML validity. So I don't think the testbrowser subclass approach is very valuable at all.
Does this belong in zope.testbrowser or in the underlying mechanize package?
It appears to me to belong higher in the stack, not lower.
I do like the local validator approach. How will you manage that dependency?
Since Ubuntu/Debian have a nice package for installing the the W3C cgi-bin/check Perl script and configuring my local system apache, I just used that and use http://localhost/cgi-bin/check. The current zope.testbrowser patch checks the aforementioned environment variable for the URL to use to validate, so that's how the user would specify not only to do the validation but where to send the validation re1quests to. I spent a bit of time trying to figure out how to get Python to invoke the W3C cgi-bin/check Perl script directly so I could just include it in the package, but it was taking too long. Do you have any idea how I can setup enough environment to directly call a cgi-bin script directly with subprocess? Other wise I suppose I'll have to manage the dependency in documentation, "ensure you have the W3C validator available at a URL. I recommend installing it locally following the instructions at...". Any other options from anyone else. Ross
participants (5)
-
Benji York -
Brian Sutherland -
Christian Theune -
Marius Gedminas -
Ross Patterson