[Zope-dev] zope-tests - FAILED: 11, OK: 12

[Marius Gedminas]

On Thu, Mar 07, 2013 at 12:07:38PM -0500, Tres Seaver wrote:
> On 03/07/2013 11:51 AM, Stephan Richter wrote:
> > On Thursday, March 07, 2013 05:46:48 PM Marius Gedminas wrote:
> >> I'm not sure that's the right thing to do.  Perhaps page templates,
> >> when given a file on disk with \r\n, should normalize the line
> >> endings to \n?
> >> 
> >> Opinions?
> > 
> > I think we should retain EOL characters as long as clients (i.e.
> > browsers) understand them, which they do.
> 
> With template source, tools may require platform-specific line endings.
> For rendered HTML / XML, no tool / browser that wants to work with the
> 'net at large can have such expectations, which makes preserving them a
> non-use-case.
> 
> +1 to Marius' proposal to open template files with 'U' in the mode.

Was that my proposal?  If only it were that easy...

* On Windows, opening the templates in text mode should've already
  converted \r\n to \n, even without specifying 'U'.  This didn't
  happen, why?

* There are two kinds of templates: HTML ones and XML ones.  zope.tal
  uses different parsers for the different kinds: expat for XML and
  HTMLParser for HTML.

* zope.tal 4.0.0a1 opens XML templates in binary mode (which doesn't
  support 'U') because expat insists on bytes, and on Python 3 files
  opened in text mode give you Unicode.

* HTML templates are opened in text mode (which implies newline
  conversion on Windows)

* Most of the templates that caused these test failures on winbot were
  HTML templates.  They should've been normalized at this point:
  https://github.com/zopefoundation/zope.tal/blob/master/src/zope/tal/htmltalparser.py#L121

* Am I going crazy?

* Wait, PageTemplateFile doesn't use parseFile!  It reads the template
  as bytes and feeds it to the wolves^H^H^H^H^H^H page template engine
  after maybe decoding it to Unicode (for HTML templates; XML ones get
  bytes):
  https://github.com/zopefoundation/zope.pagetemplate/blob/master/src/zope/pagetemplate/pagetemplatefile.py#L79

So, I think any \r\n normalization should be done in zope.pagetemplate's
PageTemplateFile.  If we want to do \r\n normalization.

Marius Gedminas
-- 
http://pov.lt/ -- Zope 3/BlueBream consulting and development
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: Digital signature
URL: <http://mail.zope.org/pipermail/zope-dev/attachments/20130307/453a7c05/attachment.sig>