[ZODB-Dev] [OT] NoSQL

[Shane Hathaway]

Wichert Akkerman wrote:
> On 11/13/09 21:33 , Shane Hathaway wrote:
>> I've been studying how to build an enormous database based on what I
>> know.  There are an incredible number of distributed databases these
>> days, but all of them concern me in one way or another.
> 
> Can you share some of those concerns with us? I'ld be interested to hear 
> what kind of problems you see.

The best article I've found is a simple presentation and overview:

http://highscalability.com/blog/2009/11/5/a-yes-for-a-nosql-taxonomy.html

The article neatly categorizes a lot of the NoSQL databases.  It 
suggests that document stores have the right level of complexity.  Wide 
columnar stores like Cassandra could be too complicated to gain a lot of 
traction, while simpler databases might lack features we would normally 
take for granted.

In other articles, I learned about CouchDB conflict resolution.  CouchDB 
allows any conflict and it stores both conflicting values, expecting the 
application to resolve the conflict later.  Clearly, CouchDB is designed 
to solve the PDA use case: I change a contact's phone number differently 
on my PDA and my desktop, then when I sync, I click some UI button to 
indicate which is correct.  I think that sort of conflict resolution 
would cause security holes for the application I am working on, but it 
would probably work for a lot of other applications.

Current versions of CouchDB expect applications to scale using 
replication.  Replication is not a substitute for sharding.  The 
couchdb-lounge project seems to be solving that with proxies:

http://tilgovi.github.com/couchdb-lounge/

The Mongo DB guys have a pretty thorough and fair comparison of CouchDB 
and Mongo:

http://www.mongodb.org/display/DOCS/Comparing+Mongo+DB+and+Couch+DB

The bottom of the page lists use cases for MongoDB.  It says people 
building a system with "very critical" transactions should choose a 
traditional RDBMS.  That seems like reasonable advice for the 
application I'm building, except that I consider ZODB to be at least as 
reliable as an RDBMS.  (RelStorage uses a subset of RDBMS functionality 
that I have found to be reliable.)

I think that by "very critical", the MongoDB authors are referring to 
applications that must not allow conflicting updates.  Conflict 
resolution is probably my main concern with all of these new databases. 
  I have no doubts about ZODB's conflict resolution policy, while I can 
imagine a variety of different policies these other databases might 
implement.  A "four or five dimensional hash" like Cassandra might even 
have a conflict resolution policy that changes with every release.

Shane