In Amos's XML-RPC HOWTO, he writes: It would be an interesting project (hint, hint) to write an XML-RPC Method Zope Product that was smart about caching, etcetera. This would make XML-RPC available from DTML in a safe form. I can take hints, and have a need for easier access to XML-RPC. So, what should an XMLRPCMethod look like? One idea is that it has a single property, the URL for an XML-RPC server. You would set this to http://othersite/zope/ or whatever. Assuming your method was given an ID of 'rpcserv', you could then write DTML like: <!--Use the standard_html_header of another Zope site (!) --> <dtml-var "rpcserv.standard_html_header()"> <dtml-in "rpcserv.listUsers()"> ... </dtml-in> This would be better named XMLRPCServer than *Method. Alternatively, by analogy with ExternalMethods, an XML-RPC Method could pin things down to a single method; you might create one for listUsers() on a given server, another for standard_html_header() on that site, etc. I don't like this, because it means you may have lots of different method objects, and when a server moves, a lot of updating is required. Too inflexible for my taste. You could make updating server URLs easier by doing something similar to DAs and SQLMethods, having XMLRPCMethods use a given XMLRPCServer object, just as SQLMethods use a given database object. This strikes me as over-engineering the problem, though, resulting in two new object types, and is still rather inflexible. Caching is an interesting problem. Who can know whether a given XML-RPC method is returning relatively static data that's constant for several hours, or transient data that changes a lot? Only the XML-RPC server, and maybe the caller, can know this. I don't think a solution is possible until XML-RPC supports some way to signal whether a method's results are cacheable or not, so I don't think caching is possible at this point in time. -- A.M. Kuchling http://starship.python.net/crew/amk/ Aristocrats need not be rich, but they must be free, and in the modern world freedom grows rarer the more we prate about it. -- Robertson Davies, "Osbert Sitwell"
At 02:41 PM 10/26/99 -0400, you wrote:
In Amos's XML-RPC HOWTO, he writes:
It would be an interesting project (hint, hint) to write an XML-RPC Method Zope Product that was smart about caching, etcetera. This would make XML-RPC available from DTML in a safe form.
I can take hints, and have a need for easier access to XML-RPC. So, what should an XMLRPCMethod look like?
One idea is that it has a single property, the URL for an XML-RPC server. You would set this to http://othersite/zope/ or whatever. Assuming your method was given an ID of 'rpcserv', you could then write DTML like:
<!--Use the standard_html_header of another Zope site (!) --> <dtml-var "rpcserv.standard_html_header()">
<dtml-in "rpcserv.listUsers()"> ... </dtml-in>
This would be better named XMLRPCServer than *Method.
Right. This allows a very open-ended use of XML-RPC from DTML.
Alternatively, by analogy with ExternalMethods, an XML-RPC Method could pin things down to a single method; you might create one for listUsers() on a given server, another for standard_html_header() on that site, etc. I don't like this, because it means you may have lots of different method objects, and when a server moves, a lot of updating is required. Too inflexible for my taste.
Well, you could acquire the server URL. This would improve flexibility. Another important benefit of an XML-RPC Method as opposed to a Server is that it limits what you can do in DTML. This allows some folks to control the hairy stuff (like XML-RPC settings) while others just use it via DTML. The Zopish thing to do (not that I can speak to whether this is always right or not) is to wrap access to external resources like the filesystem, and databases. Probably Chris or Jim could comment on the security implications of this better that I can.
You could make updating server URLs easier by doing something similar to DAs and SQLMethods, having XMLRPCMethods use a given XMLRPCServer object, just as SQLMethods use a given database object. This strikes me as over-engineering the problem, though, resulting in two new object types, and is still rather inflexible.
Perhaps. If for example, the xml-rpc server requires basic authentication or ssl to access, then this two-object approach might make more sense. If a server is just a URL, then I agree that having an object to represent a URL is overkill.
Caching is an interesting problem. Who can know whether a given XML-RPC method is returning relatively static data that's constant for several hours, or transient data that changes a lot? Only the XML-RPC server, and maybe the caller, can know this. I don't think a solution is possible until XML-RPC supports some way to signal whether a method's results are cacheable or not, so I don't think caching is possible at this point in time.
My thoughts are that when you define an XML-RPC Method you have some idea of what you are dealing with. You could then set a sensible caching parameters. For example, cache for N hours. Of course caching is more complex, for example if the method takes arguments you might not want to cache results, or might want to cache them based on the arguments. For example, the UserLand state names method is one that should be cached indefinitely but by method argument. My guess is that a simple 'cache for N hours' setting on methods would provide much of the needed functionality with very little complexity. -Amos
Amos Latteier writes:
Well, you could acquire the server URL. This would improve flexibility.
Acquire it from where? I'm not entirely sure what you're suggesting here; a Folderish XMLServer object that contains several XMLMethod objects?
The Zopish thing to do (not that I can speak to whether this is always right or not) is to wrap access to external resources like the filesystem, and databases. Probably Chris or Jim could comment on the security implications of this better that I can.
Hrm... this is at war with my primitive urge for flexibility. On the other hand, implementing it with a single object would require implementing a __getattr__ hook, to make rpcserv.foo.whatever() work, and isn't implementing __getattr__ frowned upon when using acquisition? (Does anyone have pointers to an explanation of the problems doing this?)
My guess is that a simple 'cache for N hours' setting on methods would provide much of the needed functionality with very little complexity.
Perhaps; in the meantime, I've sent off an e-mail to the XML-RPC mailing list in an effort to raise some discussion about caching. -- A.M. Kuchling http://starship.python.net/crew/amk/ in-any-case-the-best-christmas-present-i-got-today!-ly y'rs - tim -- Tim Peters, 29 Dec 91 [First occurrence of Tim Peters's long-phrase-ly idiom.]
At 04:21 PM 10/26/99 -0400, Andrew M. Kuchling wrote:
Well, you could acquire the server URL. This would improve flexibility.
Acquire it from where? I'm not entirely sure what you're suggesting here; a Folderish XMLServer object that contains several XMLMethod objects?
No just make the server URL a property of the XMLRPC Method, say 'xmlrpc_server_url' Then you could set this property on containing Folders. This would allow XMLRPC Methods with unset server urls to acquire them.
The Zopish thing to do (not that I can speak to whether this is always right or not) is to wrap access to external resources like the filesystem, and databases. Probably Chris or Jim could comment on the security implications of this better that I can.
Hrm... this is at war with my primitive urge for flexibility. On the other hand, implementing it with a single object would require implementing a __getattr__ hook, to make rpcserv.foo.whatever() work, and isn't implementing __getattr__ frowned upon when using acquisition? (Does anyone have pointers to an explanation of the problems doing this?)
Actually its ZODB that frowns on overriding __getattr__. However, if there's any group of people who are licensed to break these kind of rules my guess is that your one of them ;-) Jim as always has the details.
My guess is that a simple 'cache for N hours' setting on methods would provide much of the needed functionality with very little complexity.
Perhaps; in the meantime, I've sent off an e-mail to the XML-RPC mailing list in an effort to raise some discussion about caching.
Great! I like Paul's idea of using standard HTTP headers for this. This solution would make it a snap on the client side. Getting Frontier and other servers to set the right HTTP headers is another story all together... -Amos
Amos Latteier writes:
No just make the server URL a property of the XMLRPC Method, say 'xmlrpc_server_url' Then you could set this property on containing Folders. This would allow XMLRPC Methods with unset server urls to acquire them.
Hmm... reasonable. I've spent the afternoon working on this, and have run into several issues; will report tomorrow if I don't find solutions...
Great! I like Paul's idea of using standard HTTP headers for this. This solution would make it a snap on the client side. Getting Frontier and other servers to set the right HTTP headers is another story all together...
Note that this assumes you're always using HTTP as the transport, not e-mail, a pipe, or some other mechanism, and people are experimenting with doing it over non-HTTP protocols. -- A.M. Kuchling http://starship.python.net/crew/amk/ Whatever you do, stamp out abuses, and love those who love you. -- Voltaire
At 20:41 26/10/99 , Andrew M. Kuchling wrote:
You could make updating server URLs easier by doing something similar to DAs and SQLMethods, having XMLRPCMethods use a given XMLRPCServer object, just as SQLMethods use a given database object. This strikes me as over-engineering the problem, though, resulting in two new object types, and is still rather inflexible.
Maybe what I do with Z Network Clients is of interest. They act as local placeholders for remote Searchable Zope objects (like Z SQL Methods). - They each have their own URL, although maybe in the future I'll add acquisition of a base URL or something. - They enforce the calling parameters, like Z SQL Methods do. You have to declare them beforehand, and the namespace and the REQUEST object are searched for them. - Caching is done the same way as Z SQL Methods, keyed on arguments given. - You can use traversal and direct traversal on them (again, just like Z SQL Methods, http://an.example/Search/id/1234/name/Martijn/displayMethod). In all respects Z Network Clients sing, dance, and act just like Z SQL Methods, except that their data doesn't come from a local DA object, but from a remote Searchable Object (you need a Z Network Enabler to make those objects available). Apart from Z SQL Methods, you can also query ZCatalog objects, ZTables, and Confera topics this way. Maybe your XML-RPC methods could work the same. If you make them stick to the Searchable Object interface, then you can also use them with the Z Network objects.. =). -- Martijn Pieters, Web Developer | Antraciet http://www.antraciet.nl | Tel: +31-35-7502100 Fax: +31-35-7502111 | mailto:mj@antraciet.nl http://www.antraciet.nl/~mj | PGP: http://wwwkeys.nl.pgp.net:11371/pks/lookup?op=get&search=0xA8A32149 ------------------------------------------
I can take hints, and have a need for easier access to XML-RPC. So, what should an XMLRPCMethod look like?
One idea is that it has a single property, the URL for an XML-RPC server. You would set this to http://othersite/zope/ or whatever. Assuming your method was given an ID of 'rpcserv', you could then write DTML like:
<!--Use the standard_html_header of another Zope site (!) --> <dtml-var "rpcserv.standard_html_header()">
<dtml-in "rpcserv.listUsers()"> ... </dtml-in>
OK, I've also needed something todo this and gone ahead and created somthing that works for me. It may or may not work for other, it may be a base for anyone that wants to extend/change it. If you are interested http://www.zope.org/Members/Benno/XMLRPCClient It is still very much development at the moment. This basically just gives you an XML Server objectwhich you can place anywhere in Zope and then use it as follows <dtml-var "some_server('multiply', 3, 4)"> Obviously this is a bit of a forced example. I would prefer the syntax <dtml-var "some_server.multiply(3, 4)"> but I have had problem overriding __getattr__ so I'll stick to the former for the moment. You can also use it just as easily to retrieve a struct over XML-RPC eg: <dtml-let x="some_server('ages')"> <dtml-in "x.keys()"> <dtml-let seq=sequence-item> <dtml-var sequence-item> is <dtml-var "x[seq]"> years old<br> </dtml-let> </dtml-in> </dtml-let> or just get a specific entry: <dtml-var "someserver('ages')['Ben']"> It should also work on arrays by using <dtml-in> on the result. If you actually want to test it on a server you can try: http://www.sesgroup.net:9080 where any of the above examples *should* work. I'm not sure that this would be the prefereed syntax for all people however it seems the most logical to my little brain so I'm going with it for the moment until a can get __getattr__ working properly. It does seem a bit overkill having the XMLRPC server as an entire object however you basically have the same type of thing with mailhost at the moment as well. Any comments gratefully accepted. Benno
Ben Leslie writes:
I would prefer the syntax <dtml-var "some_server.multiply(3, 4)"> but I have had problem overriding __getattr__ so I'll stick to the former for the moment.
Looking at the code, it seems essentially the same as mine (though I haven't actually run it to verify this), and I also ran into problems making the invocation syntax more natural using __getattr__ or __call__. So we can concentrate on figuring that out... -- A.M. Kuchling http://starship.python.net/crew/amk/ We were here before any other city that now stands. And we will sing the funeral songs that are sung for cities for them when they die. -- The role of the Necropolis Litharge, in SANDMAN #55: "Cerements"
Ben Leslie writes:
I would prefer the syntax <dtml-var "some_server.multiply(3, 4)"> but I have had problem overriding __getattr__ so I'll stick to the former for the moment.
Looking at the code, it seems essentially the same as mine (though I haven't actually run it to verify this), and I also ran into problems making the invocation syntax more natural using __getattr__ or __call__. So we can concentrate on figuring that out...
Okay can any understand how I can do this? I want to change the product so I can use the syntax <dtml-var "some_server.multiply(3, 4)"> rather than <dtml-var "some_server('multiply', 3, 4)"> Below is a snippet of code which is the closet I've come to getting this to work. Unfortunately this totally kills and persistance def __getattr__(self, name): if name in ('ages', 'add', 'multiply'): myserv = xmlrpclib.Server(self.server) return (getattr(myserv,name)) else: return Acquisition.Acquired Basically I guess what I am trying todo is test if the attribute called is a valid xml rpc method (which I am just doing hard-coded at the moment) or else do whatever Zope normally does. It is the second bit which is really starting to annoy me. I figured that is I explicitly make it acquire the attribute (as done abvove) it would force Zope to do the right thing. Unfortunately it doesn't work. Anyhelp/advice would be much appreciated. Cheers, Benno
At 02:10 AM 10/28/99 +1000, you wrote:
Below is a snippet of code which is the closet I've come to getting this to work. Unfortunately this totally kills and persistance
def __getattr__(self, name): if name in ('ages', 'add', 'multiply'): myserv = xmlrpclib.Server(self.server) return (getattr(myserv,name)) else: return Acquisition.Acquired
here's some code to do what you want: class MethodMaker(ExtensionClass.Base): def __init__(self,name): self.name=self.name def __of__(self,client): myserv = xmlrpclib.Server(client.server) return getattr(myserv,self.name) class DemoClient(XMLRPCClient): ages = MethodMaker('ages') add = MethodMaker('add') multiply = MethodMaker('multiply') Instances of DemoClient will now have ages, add, and multiply 'methods'. Of course, you don't have to put these in the class, you can put them in the instance. Just have your Client class capable of adding and removing MethodMaker objects from its instance dictionary. (Note, by the way that MethodMaker doesn't need to be persistent; it suffices for it to be storable in a persistent object.) Now, if you want to know how the above actually works, here's the short rundown: ExtensionClasses (such as Acquisition and Persistence) provide a really neat hook known as the __of__ hook. If you retrieve an ExtensionClass instance as an attribute of another ExtensionClass instance, the retrieved instance's __of__ method (if it exists) is called, passing in the object it was retrieved from. The return value of the method then replaces the original object. So, when you ask a DemoClient instance for its 'ages' attribute, the ExtensionClass machinery retrieves the MethodMaker instance, then calls its __of__ method, passing in the DemoClient instance as the 'client' parameter. The return value of the __of__ method (the 'real' XML-RPC method) is then returned as if that were what was stored in the DemoClient. When you don't need a generic __getattr__ function (or can't use one due to Persistence), but need to do some type of calculation masquerading as an attribute, use of __of__ (or the ComputedAttribute class) is usually the way to go.
At 02:10 AM 10/28/99 +1000, Ben Leslie wrote:
Below is a snippet of code which is the closet I've come to getting this to work. Unfortunately this totally kills and persistance
def __getattr__(self, name): if name in ('ages', 'add', 'multiply'): myserv = xmlrpclib.Server(self.server) return (getattr(myserv,name)) else: return Acquisition.Acquired
By the way, just as a postscript to my explanation of how to do this with __of__, keep in mind that just because an object is persistent doesn't mean it can't have non-persistent sub-objects... for example: class RPCMethodCollection: def __init__(self,server,allowed_names): self.server = server self. def __getattr__(self,name): if name in self.allowed_names: myserv = xmlrpclib.Server(self.server) return (getattr(myserv,name)) raise AttributeError,name Then have XMLRPCClient add a "methods" attribute that is an instance of RPCMethodCollection. You can now say things like: <!--#with "someserver.methods"--> <!--#call "add(2,3)"--> <!--#/with--> Which leads to my final point, namely that maybe, instead of making the client object provide all the methods, maybe from a design perspective it would make more sense to have very lightweight "XMLRPCMethod" objects which talk to an "XMLRPCServer" object. This allows you to have much more fine-grained permissions control on use of the methods, and it'll be easier to call them from DTML anyhow.
At 02:10 AM 10/28/99 +1000, Ben Leslie wrote:
Below is a snippet of code which is the closet I've come to getting this to work. Unfortunately this totally kills and persistance
def __getattr__(self, name): if name in ('ages', 'add', 'multiply'): myserv = xmlrpclib.Server(self.server) return (getattr(myserv,name)) else: return Acquisition.Acquired
By the way, just as a postscript to my explanation of how to do this with __of__, keep in mind that just because an object is persistent doesn't mean it can't have non-persistent sub-objects... for example:
class RPCMethodCollection: def __init__(self,server,allowed_names): self.server = server self.
def __getattr__(self,name): if name in self.allowed_names: myserv = xmlrpclib.Server(self.server) return (getattr(myserv,name)) raise AttributeError,name
Then have XMLRPCClient add a "methods" attribute that is an instance of RPCMethodCollection. You can now say things like:
<!--#with "someserver.methods"--> <!--#call "add(2,3)"--> <!--#/with-->
Which leads to my final point, namely that maybe, instead of making the client object provide all the methods, maybe from a design perspective it would make more sense to have very lightweight "XMLRPCMethod" objects which talk to an "XMLRPCServer" object. This allows you to have much more fine-grained permissions control on use of the methods, and it'll be easier to call them from DTML anyhow.
Phillip, thanks a lot for the advice on how to use __of__ it was a great help. I personally prefer being able to freely access any method available on a server without having to create a seperate wrapper for each one. I realise that this isn't as fine grained from a security point of view, however in the project I am currently working on I will be calling a large number methods on another server. I didn't really want to have to create a wrapper for each single one, just seems like overkill to me. Although having fine grained security may be a desirable thing. I think I will need to think about this some more :) Any way at the moment I have edited my XMLRPCClient product so that it supports "server.method(x, y)" syntax. There a couple of problems with this though. 1) You have to explicitly name the functions you want to call in this manner. 2) You cannot call nested functions in this manner ie: "betty.examples.getStateName(42)" would not work. You can still access these through the "betty('examples.getStateName', 42)" syntax though. Any comments appreciated. Benno
participants (5)
-
Amos Latteier -
Andrew M. Kuchling -
Ben Leslie -
Martijn Pieters -
Phillip J. Eby