[ZODB-Dev] Plone in P2P using Zope over DHT
Shane Hathaway
shane at hathawaymix.org
Thu Jan 6 13:24:40 EST 2011
On 01/03/2011 10:23 PM, Aran Dunkley wrote:
> Hi, I'm part of a development team who are helping an organisation to
> architect a CMS based project that they want to work in a P2P network
> rather than using a centralised web-server. We'd prefer to use an
> existing popular CMS as a starting point so that it is mature, has a
> large development community and a wide range of extensions/modules
> available.
>
>> From our initial research it seems that Plone should be more capable of
> moving in to the P2P space due to it using ZODB rather than SQL and that
> ZODB seems able to be connected to a variety of storage mechanisms. I'm
> wondering what you guys, the core developers, think of the
> practicalities of Plone in P2P, for example could ZODB use a DHT as its
> storage layer? what kind of querying is required on the DHT?
>
> We have a good budget available for this and will be developing it as a
> completely free open source component, so we'd also like to hear from
> developers who may be interested in working on the project too.
I think you're right that you can think of ZODB as little more than a
transactional key-value store. It would be very cool to find a way to
implement a DHT-based ZODB storage, but here is your challenge:
1. Each ZODB thread has its own cache of objects. Each cache must be
invalidated appropriately. This cache is the key to ZODB speed, but
historically, it has also been a major source of bugs.
2. The BTrees used in catalogs are very sensitive to consistency
violations, so unless you are going to redesign the catalog, there is no
room for error in transactional consistency.
3. MVCC support is required, meaning that database reads must see old
data, not see newly committed data, until transaction boundaries.
4. Also due to the catalog, latency is a major concern. It is not
uncommon for a catalog request to read the state of 10,000 objects or
more, and you don't want that request to take more than a few seconds.
Memcached can help here.
Shane
More information about the ZODB-Dev
mailing list