On 11/27/2012 09:04 AM, Wolfgang Schnerring wrote:

Hi,

* Alex Leach <albl500@york.ac.uk> [2012-11-25 20:00]:

...

I was wondering if anyone has implemented a WebSockets server API using the zope toolkit? I've just submitted a blueprint on Launchpad (https://blueprints.launchpad.net/zopetoolkit-project/+spec/websockets-api), but thought it might be quicker and easier to discuss how one could do this here.

I'm not too familiar with WebSocket internals, but one thing that stuck with me is that you'll need to keep *lots* of open connections, which is only feasible with an eventloop-based server (which zope.server, for one, isn't).

Apart from that, I'm not sure what features remain, and where a proper home in the ZTK world would be.

At our company we've scheduled a project to integrate WebSockets into a large Zope3-based application for early next year, so we'll definitely will be doing *something* in that space -- we just don't know what, yet. ;)

What functionality did you have in mind that the ZTK might grow?

Wolfgang

Some investigations and trials showed about half a year ago that a Autobahn + Twisted + pyramid combination worked. YMMV -- Best regards, Adam GROSZER -- Quote of the day: "Nobody can hurt me without my permission." (Mohandas Gandhi)

Hi, Thanks for the reply. On Tuesday 27 Nov 2012 09:04:11 Wolfgang Schnerring wrote:

I'm not too familiar with WebSocket internals, but one thing that stuck with me is that you'll need to keep *lots* of open connections, which is only feasible with an eventloop-based server (which zope.server, for one, isn't).

Oh. That's an interesting point... I'm no expert either, but have seen them mentioned a lot recently, and started looking through the RFC the other day.

Apart from that, I'm not sure what features remain, and where a proper home in the ZTK world would be.

As I use grok, I had a little dig in grok's dependencies, and thought that zope.publisher.ws could be an appropriate home. I branched zope.publisher from Launchpad / bazaar and started copying stuff over from zope.publisher. [interfaces.]http. I thought that might be a nice way to start things going.

At our company we've scheduled a project to integrate WebSockets into a large Zope3-based application for early next year, so we'll definitely will be doing *something* in that space -- we just don't know what, yet. ;)

Same here really. I just thought I'd try and start playing with WebSockets, as they sound quite fun, persistent and fast.

What functionality did you have in mind that the ZTK might grow?

Good question. The main feature I thought ZTK would benefit from, is managing the handshake, and forwarding the connection to the relevant socket. I see no reason why the server endpoint needs to be a Zope application (could be a completely separate application, written in C++ for example), but Zope would still need to recognise the URL prefix (ws:// or wss://) and to connect the endpoints. In the grok world, I thought a conf file might be an appropriate way of configuring the server socket port numbers and whether to start a listener on (or forwarder for) that port. As the initial handshake is done over HTTP, Zope would mainly need a way of recognising the WebSocket request and to delegate it appropriately. In terms of public methods for a developer, this would be optional on whether Zope / grok manages the endpoint or not. If it did have control of the server- side WebSocket (e.g. localhost:9999), it would need a write method and methods for parsing WebSocket frames for Content-Length etc (see Section 5.2 of RFC6455), and to return only the "payload data". I haven't any experience with bit-wise operations, so could do with some help with parsing that first byte of the frame. Finally, I think testing will be a bit of b*tch, as Zope would also need a WebSocket client to perform the testing... Hope that helps. Cheers, Alex

Wolfgang

-- Alex Leach BSc. MRes. Department of Biology University of York York YO10 5DD United Kingdom www: http://bioltfws1.york.ac.uk/~albl500 EMAIL DISCLAIMER: http://www.york.ac.uk/docs/disclaimer/email.htm

On Tuesday 27 Nov 2012 07:04:43 Jim Fulton wrote:

Actually, it is. zope.server is based on asyncore.

Having said that, my impression is that web sockets is largely (or completely) orthogonal to WSGI and the HTTP-based publishing infrastructure.

From the RFC:

The WebSocket Protocol attempts to address the goals of existing bidirectional HTTP technologies in the context of the existing HTTP infrastructure; as such, it is designed to work over HTTP ports 80 and 443 as well as to support HTTP proxies and intermediaries, even if this implies some complexity specific to the current environment. However, the design does not limit WebSocket to HTTP, and future implementations could use a simpler handshake over a dedicated port without reinventing the entire protocol. This last point is important because the traffic patterns of interactive messaging do not closely match standard HTTP traffic and can induce unusual loads on some components.

ZTK is only relevant to the extent that it's a bag of useful libraries maintained by the Zope community.

It is a bag of useful libraries :) Is there anything that would ease the development of a WebSockets server, though? I would have thought there is, and that it would be at least possible, if not simple, to implement from existing base classes... KR, Alex

Hi, just sending again to the list... On Tuesday 27 Nov 2012 16:51:13 Andreas Jung wrote:

Use the right tool for each problem. Zope is really not the right choice for implementing a websocket server.

Why is it the wrong tool for the job? I read up a bit on asyncore, and see it's supposedly a single threaded server. Is that part of the problem? The way I see it, zope's got its own web server and framework that handles HTTP requests; WebSockets initially connect over HTTP, before upgrading to a WebSocket connection. If the 'upgrade' request was recognised by zope, then couldn't it just forward the connection on to any WebSocket server? It wouldn't even need to do the handshake, but a compatible proxy would no doubt need to be implemented.

There are dozens of better solutions for Python like

http://autobahn.ws/

The dependencies of all other Python-based WebSocket implementations all require a completely different application stack. Twisted, which autobahn uses, looks very powerful for low-level networking and the such, but it's a beast and I've never used it before, in favour of making templated web apps with grok.. Also, wouldn't the front-end socket of autobahn conflict with any other running HTTP server? Sure, I could set it up on an alternative port, and use a front-end webserver to proxy requests appropriately, based on URL paths and patterns, like I do now with grok and Apache. But that adds another two complications, when it comes to initially setting up the server. Installation instructions for potential users would become even more off-putting than they already are. The zope server already has a good system for URL parsing and dispatching. Would zope.proxy be any good at proxying WebSocket requests based on the URL?

Don't try to misuse a framework for a purpose that it has not been invented for.

WebSockets hadn't been invented back then, but HTTP was always meant to be bidirectional. Get with the times :b (jk) So, I think I understand that zope might not be great for managing simultaneous bidirectional traffic, which is basically a requirement of a WebSocket server. Could it not be a good proxy either? KR, Alex