[Zope] pickle passing over a socket

[Janko Hauser]

Am 22.08.2008 um 19:43 schrieb David Bear:

>
>
> On 8/22/08, Andrew Milton <akm at theinternet.com.au> wrote: +------- 
> [ David Bear ]----------------------
>
> | It seems to me that it could be usefull to have a zope method for  
> passing
> | python pickles to external processes through a unix domain socket.  
> My thinking
> | was to use this as a means to abstract a data base connection.  
> Yes, I know
> | there are zsql methods, sqlalchemy and other products to do this.  
> But my
> | thought was to have a generilzed method for just passing a pickle  
> to an
> | external process that could do anything with it. Perhaps it could  
> pass a pickle
> | to a queue that would handle the pickle if the site need some high  
> volume
> | handling of data.
> |
> | Then one could have another python process listening on the socket  
> -- ready to
> | receive pickles for zope.
>
>
> Isn't this called ZEO? d8)
>
>
> ZEO could be a model for this - but ZEO stores the pickles to zodb.  
> What if I wanted to queue the pickle up and take some kind of long  
> term processing action on it? Store it in a network file system --  
> which may have a high write latency to, put it in a data base for  
> which there were no zope connectors, etc.

Queuing, async long term processing, eventsystem are very different  
requirements. They have only a small part on the protcoll level, the  
more complex part is the logic at the recieving end. And it is easier,  
to call remote procedures or methods with local data, than to push  
pickles down a socket and decide what to do with them.

There are a number of packages wich do that and much more at a higher  
level. I would look at

http://pyro.sourceforge.net/
http://foolscap.lothar.com/trac
http://threekong.com/

On the protcoll level, there are many modern contributions from the  
bigheads at google and facebook, like Protobufs, thrift, amqp (http://barryp.org/software/py-amqplib/ 
)

HTH,

__Janko


-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 163 bytes
Desc: This is a digitally signed message part
Url : http://mail.zope.org/pipermail/zope/attachments/20080822/382ed0d7/attachment.bin