[ZODB-Dev] relstorage - memcached - noSQL(?)

[Adam GROSZER]

Hello Shane,

This is some tinkering about whether/how it is possible to make a ZODB
storage that uses a simple kev-value database (KVDB) (e.g. tokyo tyrant).
A KVDB that is persistent on disk or even replicated should not be
less fault tolerant as MySQL.

As I see you got the caching right with memcached in relstorage. That
is also a step further on this direction.

Problems that I see:
- read/write separation: as the KVDB does not have transactions, reads
  should not overlap with writes. That should easily be possible with
  locking. (Yah, that does not make it so shiny anymore.)
  Either DB global or fine-grained on the oid level.
  That would maybe require a separate service that does the locking?

- loadBefore: it's a good question how to solve that easily/speedy without
  lots of requests to the KVDB with the KVDB's reduced set of
  features.

- performance: my fear is that getting this right could be slower as
  relstorage/ZEO. Because of the mismatches between the requirements of a
  ZODB storage and the features of the simple KVDB.

There is also an S3 storage out there, by Laurence:
http://code.google.com/p/s3storage/

Jim is working on zc.bsddbstorage, where BerkeleyDB should be
(sofar I see -- but never used it) also a (not that minimal) KVDB.



I'd like to personally chase these ideas, but I don't think I'll get
some time soon. Also I guess you have a deeper knowledge about
storages.

-- 
Best regards,
 Adam GROSZER                          mailto:agroszer at gmail.com
--
Quote of the day:
Baseball is a skilled game.  It's America's game -- it, and high taxes.