[ZODB-Dev] Plone in P2P using Zope over DHT
Aran Dunkley
aran at organicdesign.co.nz
Tue Jan 4 06:40:34 EST 2011
Thanks for the feedback Vincent :-) it sounds like NEO is pretty close
to being SQL-free. As one of the NEO team, what are your thoughts on the
practicality of running Plone in a P2P environment with the latencies
experienced in standard DHT (such as for example those based on
Kademlia) implemtations?
On 04/01/11 22:27, Vincent Pelletier wrote:
> Hi.
>
> Le mardi 4 janvier 2011 07:18:34, Aran Dunkley a écrit :
>> The problem is that it uses SQL for its indexing queries (they quote
>> "NoSQL" as meaning "Not only SQL"). SQL cannot work in P2P space, but
>> can be made to work on server-clusters.
>
> Yes, we use MySQL, and it bites us on both worlds actually:
> - in relational world, we irritate developers as we ask questions like "why
> does InnoDB load a whole row when we just select primary key columns", which
> ends up with "don't store blobs in mysql"
> - in key-value world, because NoSQL using MySQL doesn't look consistent
>
> So, why do we use MySQL in NEO ?
> We use InnoDB as an efficient BTree implementation, which handles persistence.
> We use MySQL as a handy data definition language (NEO is still evolving, we
> need an easy way to tweak table structure when a new feature requires it), but
> we don't need any transactional isolation (each MySQL process used for NEO is
> accessed by only one process through one connection).
> We want to stop using MySQL & InnoDB in favour of leaner-and-meaner back-ends.
> I would especially like to try kyoto cabinet[1] in on-disk BTree mode, but it
> requires more work than the existing MySQL adaptor and there are more urgent
> tasks in NEO.
>
> Just as a proof-of-concept, NEO can use a Python BTree implementation as an
> alternative (RAM-only) storage back-end. We use ZODB's BTree implementation,
> which might look surprising as it's designed to be stored in a ZODB... But
> they work just as well in-RAM, and that's all I needed for such proof-of-
> concept.
>
> [1] http://fallabs.com/kyotocabinet/
>
> Regards,
More information about the ZODB-Dev
mailing list