[I'm copying this to the Zope list so that someone there will be sure to correct any gross mistakes I make.] Transfering binary data through XML-RPC does not seem to be the problem that you described. xmlrpclib handles it fine. Zope receives it fine. It's just unusable once it gets to Zope. As a test, I created a script that made a Binary object from a simple image foo = xmlrpclib.Binary(img.read()) and then called a test script on my Zope server. result = Zopesvr.test.xmlrpc(foo) This worked o.k. When I printed the string rendition of what the script received, I got <xmlrpclib.Binary instance at 0x98420b4> Looks good, no? This is much better than we'd thought; there is no need to manually base64 encode it on the client. However...it turns out that the Binary class in Zope's xmlrpclib product is quite a kludge. # # binary data wrapper (NOTE: this is an extension to Userland's # XML-RPC protocol! only for use with compatible servers!) class Binary: def __init__(self, data=None): self.data = data def decode(self, data): import base64 self.data = base64.decodestring(data) def encode(self, out): import base64, StringIO out.write("<value><base64>\n") base64.encode(StringIO.StringIO(self.data), out) out.write("</base64></value>\n") The first thing to notice is that we don't know what type of data is in a Binary object. You just initialize it with some data and then you either encode() or decode() it. If it was initialized with encoded data and you encode() it or it was initialized with decoded data and you decode() it, you'll probably not get what you expected. The more important deficiency in this class is that even if you do figure out what kind of data is in there, there's no way for you (as a normal user) to get at it. Both decode() and encode() want to stuff their results in a file object. I do not know of any way you can get one in Zope. It does not matter though - you are not allowed to access them anyway. So...you're screwed. You can't touch the data. (No, you can't just use a Binary object with manage_addFile.) The quick and dirty solution is to add import AccessControl AccessControl.allow_class(Binary) to xmlrpclib.py. Then you can get straight at the "data" of the Binary object. (I just did this and passed it to manage_addImage(). It worked fine. The data was already decoded.) Ideally, you'd add some sane public methods to the Binary class. Perhaps "encoded()" and "decoded()" would be a place to start. They could return either strings or StringIO objects. Sorry I don't have a better solution, but it looks like you're probably going to have to patch Zope if you don't want to give up on XML-RPC and just use HTTP POST. I appreciate and share your desire to use XML-RPC with Zope, but until we get Binary straightened out in Zope and basic authentication added in Python, it just takes too many kludges to recommend. --kyler
I'd just like to add that solutions ranging from "create your own product", "use an external method", and "look at allowing things from RestrictedPython" are not solutions. Assume that I (you, whomever) has no access to the Zope backend. The file data needs to somehow be available to a script in Zope, or a way to decode it must be available. Someone on #zope said that binary files weren't handled due to security reasons. I'm curious as to what the implications are by allowing a binary file xmlrpc upload. We are just talking about decoding the file and then creating, say, an Image or File object out of it. Or maybe we'll process it it, but I don't see the problem there either. Assuming that Python itself has no holes in it, and assuming that the decode of the file, and also the mechanisms by which Zope stores the new Image/File object are fine, I don't see the problem. (Sorry for the run-on.) Please enlighten me. -Chris On Mon, 9 Sep 2002 14:32:45 -0500, Kyler Laird spoke forth:
[I'm copying this to the Zope list so that someone there will be sure to correct any gross mistakes I make.]
Transfering binary data through XML-RPC does not seem to be the problem that you described. xmlrpclib handles it fine. Zope receives it fine. It's just unusable once it gets to Zope.
As a test, I created a script that made a Binary object from a simple image foo = xmlrpclib.Binary(img.read()) and then called a test script on my Zope server. result = Zopesvr.test.xmlrpc(foo) This worked o.k. When I printed the string rendition of what the script received, I got <xmlrpclib.Binary instance at 0x98420b4>
Looks good, no? This is much better than we'd thought; there is no need to manually base64 encode it on the client.
However...it turns out that the Binary class in Zope's xmlrpclib product is quite a kludge. # # binary data wrapper (NOTE: this is an extension to Userland's# XML-RPC protocol! only for use with compatible servers!)
class Binary:
def __init__(self, data=None): self.data = data
def decode(self, data): import base64 self.data = base64.decodestring(data)
def encode(self, out): import base64, StringIO out.write("<value><base64>\n") base64.encode(StringIO.StringIO(self.data), out) out.write("</base64></value>\n")
The first thing to notice is that we don't know what type of data is in a Binary object. You just initialize it with some data and then you either encode() or decode() it. If it was initialized with encoded data and you encode() it or it was initialized with decoded data and you decode() it, you'll probably not get what you expected.
The more important deficiency in this class is that even if you do figure out what kind of data is in there, there's no way for you (as a normal user) to get at it. Both decode() and encode() want to stuff their results in a file object. I do not know of any way you can get one in Zope. It does not matter though - you are not allowed to access them anyway.
So...you're screwed. You can't touch the data. (No, you can't just use a Binary object with manage_addFile.)
The quick and dirty solution is to add import AccessControl AccessControl.allow_class(Binary) to xmlrpclib.py. Then you can get straight at the "data" of the Binary object. (I just did this and passed it to manage_addImage(). It worked fine. The data was already decoded.)
Ideally, you'd add some sane public methods to the Binary class. Perhaps "encoded()" and "decoded()" would be a place to start. They could return either strings or StringIO objects.
Sorry I don't have a better solution, but it looks like you're probably going to have to patch Zope if you don't want to give up on XML-RPC and just use HTTP POST. I appreciate and share your desire to use XML-RPC with Zope, but until we get Binary straightened out in Zope and basic authentication added in Python, it just takes too many kludges to recommend.
--kyler
-- -------------------------------------------------------------------- Christopher N. Deckard | Lead Web Systems Developer cnd@ecn.purdue.edu | Engineering Computer Network http://eng.purdue.edu/ECN/ | Purdue University ---- zlib.decompress('x\234K\316Kq((-J)M\325KM)\005\000)"\005w') ---
This would appear to be a bug. I would file a collector issue re this. -Casey On Monday 09 September 2002 03:32 pm, Kyler Laird wrote:
[I'm copying this to the Zope list so that someone there will be sure to correct any gross mistakes I make.]
Transfering binary data through XML-RPC does not seem to be the problem that you described. xmlrpclib handles it fine. Zope receives it fine. It's just unusable once it gets to Zope.
As a test, I created a script that made a Binary object from a simple image foo = xmlrpclib.Binary(img.read()) and then called a test script on my Zope server. result = Zopesvr.test.xmlrpc(foo) This worked o.k. When I printed the string rendition of what the script received, I got <xmlrpclib.Binary instance at 0x98420b4>
Looks good, no? This is much better than we'd thought; there is no need to manually base64 encode it on the client.
However...it turns out that the Binary class in Zope's xmlrpclib product is quite a kludge. # # binary data wrapper (NOTE: this is an extension to Userland's # XML-RPC protocol! only for use with compatible servers!)
class Binary:
def __init__(self, data=None): self.data = data
def decode(self, data): import base64 self.data = base64.decodestring(data)
def encode(self, out): import base64, StringIO out.write("<value><base64>\n") base64.encode(StringIO.StringIO(self.data), out) out.write("</base64></value>\n")
The first thing to notice is that we don't know what type of data is in a Binary object. You just initialize it with some data and then you either encode() or decode() it. If it was initialized with encoded data and you encode() it or it was initialized with decoded data and you decode() it, you'll probably not get what you expected.
The more important deficiency in this class is that even if you do figure out what kind of data is in there, there's no way for you (as a normal user) to get at it. Both decode() and encode() want to stuff their results in a file object. I do not know of any way you can get one in Zope. It does not matter though - you are not allowed to access them anyway.
So...you're screwed. You can't touch the data. (No, you can't just use a Binary object with manage_addFile.)
The quick and dirty solution is to add import AccessControl AccessControl.allow_class(Binary) to xmlrpclib.py. Then you can get straight at the "data" of the Binary object. (I just did this and passed it to manage_addImage(). It worked fine. The data was already decoded.)
Ideally, you'd add some sane public methods to the Binary class. Perhaps "encoded()" and "decoded()" would be a place to start. They could return either strings or StringIO objects.
Sorry I don't have a better solution, but it looks like you're probably going to have to patch Zope if you don't want to give up on XML-RPC and just use HTTP POST. I appreciate and share your desire to use XML-RPC with Zope, but until we get Binary straightened out in Zope and basic authentication added in Python, it just takes too many kludges to recommend.
--kyler
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
On Tue, Sep 10, 2002 at 10:53:18AM -0400, Casey Duncan wrote:
This would appear to be a bug.
I'll let Chris do that, but here's my recommendation... The root of the problem is the Binary class. It might work just fine for its intended purpose, but it needs to be rewritten if anyone else is going to use it. It also needs to be accessible to restricted code. Everything could be solved if Binary were made to store its data unencoded/decoded and present it either encoded or decoded. Ideally, it could also provide whatever methods (seek and read?) are required for manage_addFile/ manage_addImage to accept it as source data ("file"). --kyler
Is there any reason to expose the encoded binary to the application at all? Why not just have the xmlrpc module decode the incoming data and just pass it along in a string or StringIO object automatically? Perhaps the latter would be better so that the decode could wait until read was actually called, thereby saving server cpu cycles if the binary is never read for some reason (like a validation error or other exception). -Casey On Tuesday 10 September 2002 12:43 pm, Kyler Laird wrote:
On Tue, Sep 10, 2002 at 10:53:18AM -0400, Casey Duncan wrote:
This would appear to be a bug.
I'll let Chris do that, but here's my recommendation...
The root of the problem is the Binary class. It might work just fine for its intended purpose, but it needs to be rewritten if anyone else is going to use it. It also needs to be accessible to restricted code.
Everything could be solved if Binary were made to store its data unencoded/decoded and present it either encoded or decoded. Ideally, it could also provide whatever methods (seek and read?) are required for manage_addFile/ manage_addImage to accept it as source data ("file").
--kyler
On Tue, Sep 10, 2002 at 12:57:53PM -0400, Casey Duncan wrote:
Is there any reason to expose the encoded binary to the application at all? Why not just have the xmlrpc module decode the incoming data and just pass it along in a string or StringIO object automatically?
That's fine with me. *I* don't have any need for the encoded version, but I can imagine someone might. If there's no reason to protect it, my tendancy is to make it available. My bigger concern was that the current class does not define what data is stored in its "data". It could be either encoded or decoded. It's up to the initializer to remember what was put in it. When it's Zope doing the initialization and a user trying to use the object, that's not good.
Perhaps the latter would be better so that the decode could wait until read was actually called, thereby saving server cpu cycles if the binary is never read for some reason (like a validation error or other exception).
Yup, that's what I was thinking also. It can always be deliberately made into a string. I can imagine this is also a more efficient way of running it through manage_addFile. --kyler
participants (3)
-
Casey Duncan -
Christopher N. Deckard -
Kyler Laird