[ZPublisher] specifiying 'charset' for the content-type header
The following usecase is not handled correctly by Zope: When you call a PythonScript returning a Python unicode string then the ZPublisher/HTTPResponse code will convert the unicode response string using HTTPResponse.default_encoding to a Python string. The default encoding is configurable through zope.conf (zpublisher_default_encoding key). This works fine unless you change the encoding to utf-8. In this case Zope sends out UTF8 content without specifying the encoding in the content-type header. I propose the following changes: - Zope adds content-type: text/plain|html when no content-type is specified by the application. In this case we would add 'charset=<DEFAULT_ENCODING> - if the application specfies a content-type header then we would add the charset only if it is undefined Objections? Andreas
Andreas Jung wrote:
The following usecase is not handled correctly by Zope:
When you call a PythonScript returning a Python unicode string then the ZPublisher/HTTPResponse code will convert the unicode response string using HTTPResponse.default_encoding to a Python string. The default encoding is configurable through zope.conf (zpublisher_default_encoding key). This works fine unless you change the encoding to utf-8. In this case Zope sends out UTF8 content without specifying the encoding in the content-type header.
I propose the following changes:
- Zope adds content-type: text/plain|html when no content-type is specified by the application. In this case we would add 'charset=<DEFAULT_ENCODING>
- if the application specfies a content-type header then we would add the charset only if it is undefined
Objections?
This sounds worryingly DWIM-y :-S How will Zope know when _not_ to add the content-type header? How will Zope tell if the charset is "undefined" and what does "undefined" mean in this context? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
--On 21. Dezember 2005 18:07:49 +0000 Chris Withers <chris@simplistix.co.uk> wrote:
How will Zope know when _not_ to add the content-type header?
Makes no sense to me. Either the application set the content-type header or Zope does it for you. Check HTTPResponse.py.
How will Zope tell if the charset is "undefined" and what does "undefined" mean in this context?
The HTTP spec says that a http server *can* set the charset to make the encoding of the payload clear. That's what we are doing now. Not specifying the charset means for the browser: *guessing* the encoding which means the browser defaults to some unspecified default encoding. -aj
Andreas Jung wrote:
How will Zope know when _not_ to add the content-type header?
Makes no sense to me. Either the application set the content-type header or Zope does it for you. Check HTTPResponse.py.
OK.
How will Zope tell if the charset is "undefined" and what does "undefined" mean in this context?
The HTTP spec says that a http server *can* set the charset to make the encoding of the payload clear. That's what we are doing now. Not specifying the charset means for the browser: *guessing* the encoding which means the browser defaults to some unspecified default encoding.
Yes, but we're essentially guessing too, right? :-S Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
--On 23. Dezember 2005 19:28:52 +0000 Chris Withers <chris@simplistix.co.uk> wrote:
The HTTP spec says that a http server *can* set the charset to make the encoding of the payload clear. That's what we are doing now. Not specifying the charset means for the browser: *guessing* the encoding which means the browser defaults to some unspecified default encoding.
Yes, but we're essentially guessing too, right? :-S
No. Imagine a PythonScript that returns a unicode string. The ZPublisher converts the unicode string to latin1 (zpublisher_default_encoding). Most browsers will default Latin1 when they can't find the charset in the content-type header. When you configure the ZPublisher to convert output to utf-8 then browsers will still default to Latin1 although the output it utf-8. So we tell the browser which encoding the ZPubisher was using to produce the output. -aj
Andreas Jung wrote:
No. Imagine a PythonScript that returns a unicode string. The ZPublisher converts the unicode string to latin1 (zpublisher_default_encoding). Most browsers will default Latin1 when they can't find the charset in the content-type header. When you configure the ZPublisher to convert output to utf-8 then browsers will still default to Latin1 although the output it utf-8. So we tell the browser which encoding the ZPubisher was using to produce the output.
Ah, okay, and how would I indicate in my PythonScript return that I'm returning something different to what is specified in zpublisher_default_encoding? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
--On 4. Januar 2006 11:23:09 +0200 Chris Withers <chris@simplistix.co.uk> wrote:
Andreas Jung wrote:
No. Imagine a PythonScript that returns a unicode string. The ZPublisher converts the unicode string to latin1 (zpublisher_default_encoding). Most browsers will default Latin1 when they can't find the charset in the content-type header. When you configure the ZPublisher to convert output to utf-8 then browsers will still default to Latin1 although the output it utf-8. So we tell the browser which encoding the ZPubisher was using to produce the output.
Ah, okay, and how would I indicate in my PythonScript return that I'm returning something different to what is specified in zpublisher_default_encoding?
By setting the content-type header _with_ a 'charset' set. -aj
Andreas Jung wrote:
Ah, okay, and how would I indicate in my PythonScript return that I'm returning something different to what is specified in zpublisher_default_encoding?
By setting the content-type header _with_ a 'charset' set.
A cool :-) Where's the code that checks if this header is present so I can take a look? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
--On 19. Januar 2006 18:37:35 +0000 Chris Withers <chris@simplistix.co.uk> wrote:
Andreas Jung wrote:
Ah, okay, and how would I indicate in my PythonScript return that I'm returning something different to what is specified in zpublisher_default_encoding?
By setting the content-type header _with_ a 'charset' set.
A cool :-) Where's the code that checks if this header is present so I can take a look?
ZPublisher/HTTPResponse.py (where else :-)) -aj
Andreas Jung wrote:
--On 19. Januar 2006 18:37:35 +0000 Chris Withers <chris@simplistix.co.uk> wrote:
Andreas Jung wrote:
Ah, okay, and how would I indicate in my PythonScript return that I'm returning something different to what is specified in zpublisher_default_encoding?
By setting the content-type header _with_ a 'charset' set.
A cool :-) Where's the code that checks if this header is present so I can take a look?
ZPublisher/HTTPResponse.py (where else :-))
Oh :-S I was expecting a different patch... so this only affects content returned as XML? How come? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
--On 20. Januar 2006 10:03:03 +0000 Chris Withers <chris@simplistix.co.uk> wrote: set' set.
A cool :-) Where's the code that checks if this header is present so I can take a look?
ZPublisher/HTTPResponse.py (where else :-))
Oh :-S
I was expecting a different patch... so this only affects content returned as XML? How come?
huh? -aj
Andreas Jung wrote:
A cool :-) Where's the code that checks if this header is present so I can take a look?
ZPublisher/HTTPResponse.py (where else :-))
Oh :-S
I was expecting a different patch... so this only affects content returned as XML? How come?
huh?
Sorry, my mistake, looking at the wrong diff... Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
participants (2)
-
Andreas Jung -
Chris Withers