[ZODB-Dev] Plone in P2P using Zope over DHT

[Vincent Pelletier]

Hi.

Le mardi 4 janvier 2011 07:18:34, Aran Dunkley a écrit :
> The problem is that it uses SQL for its indexing queries (they quote
> "NoSQL" as meaning "Not only SQL"). SQL cannot work in P2P space, but
> can be made to work on server-clusters.

Yes, we use MySQL, and it bites us on both worlds actually:
- in relational world, we irritate developers as we ask questions like "why
  does InnoDB load a whole row when we just select primary key columns", which
  ends up with "don't store blobs in mysql"
- in key-value world, because NoSQL using MySQL doesn't look consistent

So, why do we use MySQL in NEO ?
We use InnoDB as an efficient BTree implementation, which handles persistence.
We use MySQL as a handy data definition language (NEO is still evolving, we 
need an easy way to tweak table structure when a new feature requires it), but 
we don't need any transactional isolation (each MySQL process used for NEO is 
accessed by only one process through one connection).
We want to stop using MySQL & InnoDB in favour of leaner-and-meaner back-ends.
I would especially like to try kyoto cabinet[1] in on-disk BTree mode, but it 
requires more work than the existing MySQL adaptor and there are more urgent 
tasks in NEO.

Just as a proof-of-concept, NEO can use a Python BTree implementation as an 
alternative (RAM-only) storage back-end. We use ZODB's BTree implementation, 
which might look surprising as it's designed to be stored in a ZODB... But 
they work just as well in-RAM, and that's all I needed for such proof-of-
concept.

[1] http://fallabs.com/kyotocabinet/

Regards,
-- 
Vincent Pelletier