[Zope-dev] [ZODB-Dev] [announce] NEO 1.0 - scalable and redundant storage for ZODB

[Vincent Pelletier]

On Mon, 27 Aug 2012 14:37:37 +0200,
Vincent Pelletier <vincent at nexedi.com> wrote :
> Under the hood, it relies on simple features of SQL databases

To make things maybe a bit clearer, from the feedback I get:
You can forget about SQL presence. NEO usage of SQL is as a relational
as a handful of python dicts is. Except there is no way to load only
part of a pickled dict, or do range searches (ZODB's BTrees are much
better in this regard), or writable to disk atomically without having to
implement this level of atomicity ourselves.

Ideally, NEO would use something like libhail, or maybe even simpler
like kyotocabinet (except that we need composed keys, and kyotocabinet
b-trees have AFAIK no such notion).
SQL as a data definition language was simply too convenient during
development (need a new column ? easy, even if you have a 40GB table),
and it stuck - and we have yet to find a significant drawback to
implement a new storage backend.

As a side effect, SQL allows gathering some statistics over the data
contained in a database very efficiently. Number of current objects,
number of revisions per object, number of transactions, when
transactions occured in base history, average object size, largest
object, you name it.

-- 
Vincent Pelletier
ERP5 - open source ERP/CRM for flexible enterprises