Re: [Zope-dev] Using Zope in a client-server system
Tres Seaver wrote:
Michel Pelletier wrote:
Itai Tavor wrote: <snip>
There are a couple of features that I am not sure how to implement though:
- I need to display constantly-updating information on the workstations. I could do this by reloading the window every second, but that seems ugly - I'd like to only update certain fields, which I can do using layers and JavaScript, but I need to get the updated data from the server. I guess that can be done with Java, but I'd like to keep the system simple - a browser with HTML and JavaScript only on the client side, DTML, Zope with Python Methods, and External Methods, and an SQL database on the server, no Java or any other complications.
Ah, well, you see you've hit one of the disadvantages of using a stateless protocol like HTTP to solve your problem. There is no good general purpose solution that I know of to address this issue.
HTTP not being bi-directional is a bigger problem than it being stateless... I planned to use a session product that will identify each workstation and keep a record of exactly what is happening on it. But the biggest problem is that the browser / web server setup is client event driven.
- I also need to push data to the workstations - based on events detected at the server, I need to bring up a message on a workstation, and remove it in response to another event. I really don't want to refresh a frame every second for this, and I also would hate to have to write a Java applet to do it.
This is the identical problem to the one above.
Any comments or suggestions regarding the above problems, the choice of Zope for this project, or the project in general would be appreciated.
If you want to go far enough, you can try and use XML-RPC (also, stateless, but not client-bound to a certain model) and something like wxPython to build your own GUI that can at least mantain some sort of persistent or bi-directional communication between your clients and servers. But this is a big task.
Programming my own GUI is already farther than I'd like to go... and if I do, would it still be a good idea to use Zope on the server side? If I use Zope, wouldn't I still have to use HTTP? How otherwise can I get the client to communicate with the server? (Complete ignorance of XML-RPC implied here.)
I think if you maybe rethink your problem you can compromise on something that does not require the server pushing data to the client.
Yeah, without actually failing to deliver any of the features the client is expecting... the only way I can think of is simply to reload certain frames every second... seems like a very ugly solution.
I need to quote on time and cost for this project, and to do that I have to figure out the tools I'd be using. I'm sure a Zope/web browser solution will be the quickest, but if I go that way and get stuck, I won't be in a very enviable position. I'm sure that most programmers would immediately start coding Java or VB at this point... am I making a mistake trying to apply Zope to a task it's not good for?
By default, browsers aren't servers: they don't know from bind()/listen()/accept(), which is what push technology requires (unless you have a long-running, bi-directional connection between browser and server, which HTTP doesn't allow). Polling is the only "native" technique for refresh, and suffers from _horrible_ scalability problems.
Options in order of simplicity:
1. Pull instead of push (ick!)
My sentiments exactly.
2. As Hung Jung suggests, build the "push model" stuff separately from the browser, using Python or Java to implement HTTP (and perhaps XML-RPC?) Integration with the browser is difficult: clients will have to have some mechanism for registering their mini-server with your server; if you want the broser's display to update when the push event comes in, you may be stuck.
Right. Mixing two GUI's - a browser for most of the interface and a separate app for pushed data - would be real ugly. So I either have to work inside the browser, as in option 3 below, or code my own GUI. Registering the mini-servers would not be a problem. This project will be running on a small number of workstations, all located in one location. It's not designed for public or wide-area use and won't even be connected to the internet. So, the problem with this solution is how to get the pushed data into the GUI.
3. Code a lightweight socket server applet in Java, persuade everyone in the world to trust you to run it in their "applet sandbox", and then "register" it with your server.
Again, because of the physical layout of the project, trust won't be an issue, which makes this solution rather attractive.
4. Write a CORBA callback object in Java, and register it with your server; the server then pushes events to your object (scalability is again an issue, as the server blocks for each client being pushed). Note that adding CORBA into Zope is a non-trivial exercise, for the moment at least; see my "notes":http://www.zope.org/Members/tseaver/CosNaming on the issues.
I read your notes, and I think I'd like to stay away from COBRA - I'd use it if I decided to go totally overboard and code both the server and client in Java. But with Zope on the server end, XML-RPC seems like the best way to communicate.
5. Write a CORBA event-channel applet (CosEventPushConsumer) in Java, set up a CosEvent/CosNotification channel, subscribe the applet to it, and push events to the channel from your server (decouples your server from event-delivery hassles/blockage, scales MUCH better than direct callbacks).
Scalability is not much of a problem for me. I wouldn't like the server to block too much but that shouldn't be much of a problem either - with 10 clients on a 100baseT network, and 4 Zope threads on a fast server, I can't imagine running into any performance problems. But I do want to keep the number of technologies and separate apps involved to a minimum, and this seems fairly complicated. Out of the various options suggested it seems to me that the main options to consider are: - Use a browser for the main GUI, and either compromise the requirements to eliminate the need for push or code an applet to display push data in the browser. - Write my own GUI. The second option raises some new questions: - Would it still be a good idea to use Zope for the server, or should I code the server as well (a full Java solution comes to mind using COBRA for two-way communication). How would I handle push in Zope? Assuming I code a push client that speaks XML-RPC, how do I get Zope to send data outside the context of an http transaction? Actually, this same question is relevant for the solution of using a Java applet within the browser. - Two good candidates for writing the GUI in are Java and wxPython. Using wxPython would have the advantage of keeping everything in Python. On the other hand, there's an XML-RPC server for Java, but not for Python - I guess I would use HTTP if I use wxPython. Can anyone suggest pros/cons for these solutions? I have one other question related to the use of Zope in this project: The server has to monitor a serial line and respond to received data by pushing messages to the clients. What would be the best way to achieve this? An external python process reading the serial line and calling ZPublisher.Client to triger a DTML method that will do the rest of the work? P.S. I know this is getting way off topic for the Zope group. I really appreciate all of you contributing your experience. Itai -- Itai Tavor -- "Je sautille, donc je suis." -- itavor@vic.bigpond.net.au -- - Kermit the Frog -- -- "What he needs now is understanding... and a confederate victory" -- -- Dr. Jacobi, Twin Peaks --
If you are on a 100T-base network, just use polling in a hidden frame, like someone suggested. Keep it simple, it will be fast, and you can use Zope. It'll save you a lot of work. With either a Javascript timeout called method or a HTTP Refresh header you reload the contents of a hidden frame, that will only contain Javascript. The javascript updates all information in the visible frames, and only that information that nees updateing. Depending on how current the information eeds to be, you can set polling intervals as low as 15 seconds on such a local, fast network. No human certainly needs anything faster, you probably can get away with intervals of 5 minutes. Martijn Pieters | Software Engineer mailto:mj@digicool.com | Digital Creations http://www.digicool.com/ | Creators of Zope http://www.zope.org/ | The Open Source Web Application Server ---------------------------------------------
Possible solution - If it is possible. I have an idea. Unfortunately I don't currently know enough about the internals of HTTP to know if the idea is feasible. I hope someone on this list does and can tell me whether the idea is workable or not. If I understand correctly, a browser sends a http request to the server for a particular window or frame and then waits for the server to respond. After a certain amount of time it times out and stops waiting for a response and puts up an error message. Is there a way to allow or force a frame or window to wait indefinitely for a response to it's request? If so, could not a small window or frame be created at the beginning of a session that sends a request to the server containing info about who is now connected. The method activated by the request would log the request particulars in either ZODB or a backend db. When an event ocurs that the server needs to notify the client about it retrieves the info about the open request and uses that info to generate a response. The response could also reset the state of the window/frame to again send another request and begin waiting indefinitely. This request would be more along the lines of an aknowledgement that the client is still connected and has received the response. Does this seem possible?? Jim Sanford
Jim Sanford wrote:
Is there a way to allow or force a frame or window to wait indefinitely for a response to it's request? If so, could not a small window or frame be created at the beginning of a session that sends a request to the server containing info about who is now connected. The method activated by the request would log the request particulars in either ZODB or a backend db. When an event ocurs that the server needs to notify the client about it retrieves the info about the open request and uses that info to generate a response. The response could also reset the state of the window/frame to again send another request and begin waiting indefinitely. This request would be more along the lines of an aknowledgement that the client is still connected and has received the response.
Does this seem possible??
Netscape Server Push - introduced in ver 1.1 - seems to be what you mean: http://www.netscape.com/home/demo/1.1b1/pushpull.html Still work with version 4.7, although the animation at the above URL doesn't. Try http://www.theriver.com/trwrc/serverpush.html for a demo. No idea about Mozilla. -- -= This is NOT a pyramid scheme =- The SNAFU Principle: True communication is possible only between equals Itamar S.T. itamars@ibm.net
This request would be more along the lines of an aknowledgement that the client is still connected and has received the response.
Does this seem possible??
Netscape Server Push - introduced in ver 1.1 - seems to be what you mean: http://www.netscape.com/home/demo/1.1b1/pushpull.html
Still work with version 4.7, although the animation at the above URL doesn't. Try http://www.theriver.com/trwrc/serverpush.html for a demo. No idea about Mozilla.
I haven't read the whole thread, so forgive me if this is out of turn... A cross-browser solution might be to start a new browser window pointing to a "connectMonitor" URL on your server, that requests a client-side refresh every 10 seconds (or 10 minutes or whatever you need). This is done with a simple META tag in the <HEAD> section of the HTML doc. <META HTTP-EQUIV="Refresh" content="32;URL=http://myserver.net/connectMonitor"> Assuming 'connectMonitor' is the name of the URL serving up this page. This example makes the client wait 32 seconds and then repoll; change the number to anything you want (zero makes an immediate refresh). Easy to do, and it's standard HTML/HTTP. When you want to publish a new message to all connected users, have your 'connectMonitor' method hook a bit of Javascript on the document's onLoad() event, which will open a new browser window at the URL representing your message. (Or I guess you could just include the message in the 'connectMonitor' page if it's something that is transient.) -- Graham
Itai Tavor wrote:
Tres Seaver wrote:
By default, browsers aren't servers: they don't know from bind()/listen()/accept(), which is what push technology requires (unless you have a long-running, bi-directional connection between browser and server, which HTTP doesn't allow). Polling is the only "native" technique for refresh, and suffers from _horrible_ scalability problems.
Options in order of simplicity:
1. Pull instead of push (ick!)
My sentiments exactly.
2. As Hung Jung suggests, build the "push model" stuff separately from the browser, using Python or Java to implement HTTP (and perhaps XML-RPC?) Integration with the browser is difficult: clients will have to have some mechanism for registering their mini-server with your server; if you want the broser's display to update when the push event comes in,you may be stuck.
Right. Mixing two GUI's - a browser for most of the interface and a separate app for pushed data - would be real ugly. So I either have to work inside the browser, as in option 3 below, or code my own GUI.
Registering the mini-servers would not be a problem. This project will be running on a small number of workstations, all located in one location. It's not designed for public or wide-area use and won't even be connected to the internet.
If you control the deployment platform, and the pipe between client and server, then some of my tradeoffs don't apply, as you note: trusted applets are reasonable, for instance, and polling is not nearly as terrible a strategy over a fast network with a small number of clients.
So, the problem with this solution is how to get the pushed data into the GUI.
3. Code a lightweight socket server applet in Java, persuade everyone in the world to trust you to run it in their "applet sandbox", and then "register" it with your server.
Again, because of the physical layout of the project, trust won't be an issue, which makes this solution rather attractive.
4. Write a CORBA callback object in Java, and register it with your server; the server then pushes events to your object (scalability is again an issue, as the server blocks for each client being pushed). Note that adding CORBA into Zope is a non-trivial exercise, for the moment at least; see my "notes":http://www.zope.org/Members/tseaver/CosNaming on the issues.
I read your notes, and I think I'd like to stay away from COBRA - I'd use it if I decided to go totally overboard and code both the server and client in Java. But with Zope on the server end, XML-RPC seems like the best way to communicate.
CORBA is probably overkill, in your case; the reason I introduced it into the discussion is to lead to #5, which gives you scalable event delivery in a heterogenous, wide-area environment.
5. Write a CORBA event-channel applet (CosEventPushConsumer) in Java, set up a CosEvent/CosNotification channel, subscribe the applet to it, and push events to the channel from your server (decouples your server from event-delivery hassles/blockage, scales MUCH better than direct callbacks).
Scalability is not much of a problem for me. I wouldn't like the server to block too much but that shouldn't be much of a problem either - with 10 clients on a 100baseT network, and 4 Zope threads on a fast server, I can't imagine running into any performance problems. But I do want to keep the number of technologies and separate apps involved to a minimum, and this seems fairly complicated.
Out of the various options suggested it seems to me that the main options to consider are:
- Use a browser for the main GUI, and either compromise the requirements to eliminate the need for push or code an applet to display push data in the browser.
- Write my own GUI.
The second option raises some new questions:
- Would it still be a good idea to use Zope for the server, or should I code the server as well (a full Java solution comes to mind using COBRA for two-way communication). How would I handle push in Zope? Assuming I code a push client that speaks XML-RPC, how do I get Zope to send data outside the context of an http transaction? Actually, this same question is relevant for the solution of using a Java applet within the browser.
You might look at Loren Stafford's recently-announced ZScheduler product: Loren is working on a way to have "internally-generated" events fire inside Zope.
- Two good candidates for writing the GUI in are Java and wxPython. Using wxPython would have the advantage of keeping everything in Python. On the other hand, there's an XML-RPC server for Java, but not for Python - I guess I would use HTTP if I use wxPython. Can anyone suggest pros/cons for these solutions?
I think I missed something here: Python *does* have an XML-RPC server implementation (xmlrpclib.py), which is how Zope can do XML-RPC.
I have one other question related to the use of Zope in this project: The server has to monitor a serial line and respond to received data by pushing messages to the clients. What would be the best way to achieve this? An external python process reading the serial line and calling ZPublisher.Client to triger a DTML method that will do the rest of the work?
Right, but ot necessarily a DTML Method. The ZClient request might, for example, trigger a method of a Python product instance, which could enqueue the event for processing by a notification thread, for instance.
P.S. I know this is getting way off topic for the Zope group. I really appreciate all of you contributing your experience.
I don't think this is off-topic at all -- we are pushing the *normal* Zope envelope a bit, but I think Zope is up to the challenge. Tres. -- ========================================================= Tres Seaver tseaver@palladion.com 713-523-6582 Palladion Software http://www.palladion.com
participants (6)
-
Graham Fawcett -
Itai Tavor -
Itamar Shtull-Trauring -
Jim Sanford -
Martijn Pieters -
Tres Seaver