Any clues as to how I trace the source of this error? I get it trying to make an xmlrpc call on my Zope server. I have other xmlrpc calls that work fine. It's not hitting my Product code, that I'm sure of... Richard -- Richard Jones richard@bizarsoftware.com.au Senior Software Developer, Bizar Software (www.bizarsoftware.com.au)
richard@bizarsoftware.com.au wrote:
Any clues as to how I trace the source of this error? I get it trying to make an xmlrpc call on my Zope server. I have other xmlrpc calls that work fine.
It's not hitting my Product code, that I'm sure of...
Turns out I can trace this to: Traceback (innermost last): File "/home/zope/Zope/lib/python/xmlrpclib.py", line 531, in loads p.feed(data) File "/usr/lib/python1.5/xmllib.py", line 149, in feed self.goahead(0) File "/usr/lib/python1.5/xmllib.py", line 227, in goahead self.syntax_error('illegal character in content') File "/usr/lib/python1.5/xmllib.py", line 748, in syntax_error raise RuntimeError, 'Syntax error at line %d: %s' % (self.lineno, message) RuntimeError: Syntax error at line 34: illegal character in content I'm using the latest xmlrpclib.py from www.pythonware.com for the client side - so I'm happy to blame it for generating invalid XML. I don't know whether it should or should not be dealing with the dodgy data I'm feeding it, but I guessed from the xmlrpc "spec" that I should be safe feeding it just about anything. Would it be possible though to have Zope handle this sort of error rather than throw a "500 Internal Server Error"? Richard -- Richard Jones richard@bizarsoftware.com.au Senior Software Developer, Bizar Software (www.bizarsoftware.com.au)
richard@bizarsoftware.com.au wrote:
RuntimeError: Syntax error at line 34: illegal character in content
This was the bane of one of our projects to the point where Andy re-wrote ZMailIn to no-longer use XML-RPC. It's to do with stuff-which-should-probably-be-entities but we never tracked down what wasn't encoding something when it should have done. I'd blame the marshalling code in xmlrpclib.py myself. If you ever fix this, do let us know :-) cheers, Chris
Chris Withers wrote:
richard@bizarsoftware.com.au wrote:
RuntimeError: Syntax error at line 34: illegal character in content
This was the bane of one of our projects to the point where Andy re-wrote ZMailIn to no-longer use XML-RPC.
Yeah, I am currently base64'ing the data, and it works.
It's to do with stuff-which-should-probably-be-entities but we never tracked down what wasn't encoding something when it should have done. I'd blame the marshalling code in xmlrpclib.py myself.
If you ever fix this, do let us know :-)
Unfortunately, I have no time to go chasing around the xmlrpc specs :( Richard -- Richard Jones richard@bizarsoftware.com.au Senior Software Developer, Bizar Software (www.bizarsoftware.com.au)
Andy McKay wrote:
Yeah, I am currently base64'ing the data, and it works.
Ive seen this in some code but didnt want to say anything for fear of sounding stupid. It doesnt seem right somehow.
See, this is the problem. I'm running into crunch-time at the moment, and I really don't have time to check the validity of third-party xmlrpc code. I honestly don't know if it's valid to send arbitrary binary data in an xmlrpc transaction. Richard -- Richard Jones richard@bizarsoftware.com.au Senior Software Developer, Bizar Software (www.bizarsoftware.com.au)
richard@bizarsoftware.com.au wrote:
Andy McKay wrote:
Yeah, I am currently base64'ing the data, and it works.
Ive seen this in some code but didnt want to say anything for fear of sounding stupid. It doesnt seem right somehow.
See, this is the problem. I'm running into crunch-time at the moment, and I really don't have time to check the validity of third-party xmlrpc code. I honestly don't know if it's valid to send arbitrary binary data in an xmlrpc transaction.
Interestingly, it looks like xmlrpclib.py can *receive* base64-encoded data automagically, but it never *sends* it. ppml.py, found in Zope2/lib/python/Shared/DC/xml, has a convert() method which could serve as a guide for making a patch to xmlrpclib that will enable it to automatically encode in base64 when appropriate. The problem with sending binary data unencoded is line endings: \r\n might be "helpfully" converted to \n and vice-versa. Other characters can be encoded but apparently parsers treat and as a real line ending. In fact it's time to convert xmlrpclib to use the Expat module (via SAX) in Python 2.1. It should speed it up dramatically. Shane
Shane Hathaway wrote:
The problem with sending binary data unencoded is line endings: \r\n might be "helpfully" converted to \n and vice-versa. Other characters can be encoded but apparently parsers treat and as a real line ending.
I'm not sure where this leaves me... I think this means that ultimately it's not possible to send binary data which contains 0x10 or 0x1310 in an XML-RPC message. Bummer. Especially with the ~30% bloat incurred by base64 encoding :(
In fact it's time to convert xmlrpclib to use the Expat module (via SAX) in Python 2.1. It should speed it up dramatically.
I wish we had the luxury of moving to python 2.1 :( Richard -- Richard Jones richard@bizarsoftware.com.au Senior Software Developer, Bizar Software (www.bizarsoftware.com.au)
On Fri, 11 May 2001 richard@bizarsoftware.com.au wrote:
Shane Hathaway wrote:
The problem with sending binary data unencoded is line endings: \r\n might be "helpfully" converted to \n and vice-versa. Other characters can be encoded but apparently parsers treat and as a real line ending.
I'm not sure where this leaves me... I think this means that ultimately it's not possible to send binary data which contains 0x10 or 0x1310 in an XML-RPC message. Bummer. Especially with the ~30% bloat incurred by base64 encoding :(
I feel like the absence of base-64 encoding is an oversight in the protocol, along with the absence of a "null" type (which would correspond to "None".) That's why I think it's not a kludge to extend XML-RPC. But why is the 30% bloat an issue? The important parts of the base64 module are in C and the encoding and decoding is fast.
In fact it's time to convert xmlrpclib to use the Expat module (via SAX) in Python 2.1. It should speed it up dramatically.
I wish we had the luxury of moving to python 2.1 :(
Actually the sax / expat module is also available for Python 1.5.2, if that helps. Shane
Shane Hathaway wrote:
I feel like the absence of base-64 encoding is an oversight in the protocol, along with the absence of a "null" type (which would correspond to "None".) That's why I think it's not a kludge to extend XML-RPC.
The omission of the explicit null is one of the oldest gripes with XML-RPC (and last time I read up on it, Dave Winer was firmly against its inclusion). It's usually the first thing I add to the "spec" when I'm working with it.
But why is the 30% bloat an issue? The important parts of the base64 module are in C and the encoding and decoding is fast.
Encode/decode speed is fine (using binascii, not base64 ;) Mostly it's network bandwidth issue. We're implementing an "offline management" feature specifically for users who are on the end of a slow link to their server. We're looking at transferring potentially big images. It is therefore a high priority to make the xml-rpc transactions as fast as possible. I'm starting to look around for other mechanisms. I looked into faking file upload form submissions, but Python's support for multipart/form-data is pretty rough - and from my reading, the RFC is rather hazy on the support of unencoded data. Zope to Zope FTP is my next avenue to investigate ;)
In fact it's time to convert xmlrpclib to use the Expat module (via SAX) in Python 2.1. It should speed it up dramatically.
I wish we had the luxury of moving to python 2.1 :(
Actually the sax / expat module is also available for Python 1.5.2, if that helps.
Yeah, as long as there's pre-compiled binaries for win32, I'm happy. I'm currently in VC++ hell trying to get freetype2 and the python ft2_interface compiled for windows. Argh. Oh yeah, another thing I noticed is that the Zope profiling seems to break if an xmlrpc call is invoked. Just hangs when it hits the call. Any clues? Richard -- Richard Jones richard@bizarsoftware.com.au Senior Software Developer, Bizar Software (www.bizarsoftware.com.au)
participants (4)
-
Andy McKay -
Chris Withers -
richard＠bizarsoftware.com.au -
Shane Hathaway