[ZODB-Dev] Plone in P2P using Zope over DHT

[Matthias]

Am 04.01.2011, 12:40 Uhr, schrieb Aran Dunkley <aran at organicdesign.co.nz>:

> Thanks for the feedback Vincent :-) it sounds like NEO is pretty close
> to being SQL-free. As one of the NEO team, what are your thoughts on the
> practicality of running Plone in a P2P environment with the latencies
> experienced in standard DHT (such as for example those based on
> Kademlia) implemtations?

Something which may be worthwhile and give you an impression what the  
storage backend does for a common operation would be to instrument the  
ZODB code a bit. Just look at the current FileStorage and add a few log()s  
into its load/store methods. Maybe there are other methods of interest,  
too. Hooking this shouldn't take long.

Then I suggest you generate (or maybe you already have) a well-sized plone  
to test on. Perform a typical request on a site and see what the storage  
is doing. This will give you a solid idea what the storage has to do.

My guess (and I mostly infer this from the zodb code I've looked at) is  
that you can get many storage read requests for something like searching  
the catalog. I guess this will happen, because you get a load() call for  
each BTree bucket that you are traversing. Maybe I am totally wrong of  
course :) However, instrumenting the storage will show.

You might also find there are certain hotspots (like the catalog).  
Depending on their size you could make your users download these  
completely from the cloud before being able to use them. This would reduce  
the number of small randomly-seeking requests a lot.

Another thing you need to consider is implementing the transaction  
functionality. I'm not sure how to do something like this in the cloud or  
even if it's possible at all (given certain performance limitations).

And finally, my experiences using P2P is that it takes a while to build up  
speed for a certain download. And client uplinks are not as fast your  
typical web server's uplink. Also firewalls seem to cause problems  
sometimes (maybe workable with NAT punching?).

Maybe all of this is feasible in your situation, but this depends heavily  
on your requirements, usage and goals. Doing some fast prototyping and  
contacting authors who already wrote papers (or better implementations) is  
probably the best way to get solid a solid idea.

-Matthias