[Zope-dev] NEO High Performance Distributed Fault Tolerant ZODB Storage

[Vincent Pelletier]

Hi,

I would like to present you the NEOPPOD project, aiming at improving ZODB 
Storage scalability. The implementation is in a rather good shape, although it
fails at a few ZODB tests at the moment (they are currently being worked on).
Scalability is achieved by distributing data over multiple servers
(replication and load balancing) with the ability to extend/reduce cluster
on-line.

Its code is available under the GPL, more information can be found on the 
project website[1].

One nice aspect is that the underlying protocol is being analysed with 
model-checking tools based on Petri Nets by a team of post-doc, PhD 
students and researchers. An article should appear in PETRI NETS 2010 [2].

We hope that NEO will be usable for production systems in 12 months from 
now, and will notify the community the day we think it is, ie. after 
using it ourselves. Meanwhile, it can be interesting for research and 
fun. Contributions are very welcome (extend portability beyond Linux 2.6, 
replace MySQL daemon dependency with a lighter & embeddable transactional 
storage, etc). For now, it has been manually confirmed to run Plone and ERP5.

Of course, to get the best out of NEO, a Zope application (or ZODB-based
application) needs to be designed in a way taking advantage of back-end
parallelism (much in the same way that a single-process application cannot
take advantage of SMP). We wrote a presentation[3] out of our experience with 
ERP5 scalability testing & improvement, which might be an interesting read for 
people developing on the Zope framework. It describes what was the most common
mistakes we corrected, and tools we developed to further extend scalability at 
various levels (NEO being the latest).

And for most productions system, ZEO is really great. We have for 
example used ZEO for now more than 2 years to operate a Central Bank ERP 
with 300 concurrent users and ZEO never crashed.

[1] http://www.neoppod.org/
[2] http://acsd-petrinets2010.di.uminho.pt/
[3] http://www.myerp5.com/kb/enterprise-High.Performance.Zope/view

Note: If this mail would fit another list better, please advise.

Regards,
--
Vincent Pelletier