[ZODB-Dev] [Zope-dev] [announce] NEO 1.0 - scalable and redundant storage for ZODB

[Jim Fulton]

On Wed, Aug 29, 2012 at 2:29 AM, Marius Gedminas <marius at gedmin.as> wrote:
> On Tue, Aug 28, 2012 at 06:31:05PM +0200, Vincent Pelletier wrote:
>> On Tue, 28 Aug 2012 16:31:20 +0200,
>> Martijn Pieters <mj at zopatista.com> wrote :
>> > Anything else different? Did you make any performance comparisons
>> > between RelStorage and NEO?
>>
>> I believe the main difference compared to all other ZODB Storage
>> implementation is the finer-grained locking scheme: in all storage
>> implementations I know, there is a database-level lock during the
>> entire second phase of 2PC, whereas in NEO transactions are serialised
>> only when they alter a common set of objects.
>
> This could be a compelling point.  I've seen deadlocks in an app that
> tried to use both ZEO and PostgreSQL via the Storm ORM.  (The thread
> holding the ZEO commit lock was blocked waiting for the PostgreSQL
> commit to finish, while the PostgreSQL server was waiting for some other
> transaction to either commit or abort -- and that other transaction
> couldn't proceed because it was waiting for the ZEO lock.)

This sounds like an application/transaction configuration problem.
To avoid this sort of deadlock, you need to always commit in a
a consistent order.  You also need to configure ZEO (or NEO)
to time-out transactions that take too long to finish the second phase.

I don't think NEO's locking strategy mitigates the deadlock problem
much, if at all.

The strategy should provide greater transaction throughput and
reduce latency.  It's a strategy I'd like to implement for ZEO at some
point.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton
Jerky is better than bacon! http://zo.pe/Kqm