[ZODB-Dev] [ zodb-Bugs-550246 ] ClientStorage Deadlock problem

[noreply@sourceforge.net]

Bugs item #550246, was opened at 2002-04-29 12:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=115628&aid=550246&group_id=15628

Category: None
Group: None
Status: Open
Resolution: None
>Priority: 1
Submitted By: Chris Withers (fresh)
Assigned to: Jeremy Hylton (jhylton)
Summary: ClientStorage Deadlock problem

Initial Comment:
Shane helped me get to the cause of this on #zope...

<hathawsh> The real problem is that if you have multiple ClientStorages that point to the same 
storage on the server, and two or more are involved in the same transaction, a deadlock happens.

This is a pattern that would be very useful to me, it currently causes the storage servers and any 
clients connection to them to hang on get_transaction().commit() until they've _all_ been 
restarted...

cheers,

Chris

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2002-11-09 17:36

Message:
Logged In: YES 
user_id=6380

Is this still a problem with ZODB 3.1 and Zope 2.6?

----------------------------------------------------------------------

Comment By: Shane Hathaway (hathawsh)
Date: 2002-04-30 09:42

Message:
Logged In: YES 
user_id=16701

Chris had two ClientStorages connecting to the same storage.
As soon as he tried to write (non-conflicting) changes to
both client storages in the same transaction, the storage
server locked up.  It would be better if it caused an error.
 I took a glance at the code and couldn't find an easy way
to detect this error condition, but I'm not as familiar with
ZEO internals as Jim or Jeremy.

In the future, we will set up mounting in such a way that
one ClientStorage can be used for multiple mounts.  We
should keep this in mind for Zope 3.


----------------------------------------------------------------------

Comment By: Chris Withers (fresh)
Date: 2002-04-30 08:03

Message:
Logged In: YES 
user_id=24723

What I had was two ExternalMount objects both connecting to different folders but served from the same 
ClientStorage, just mounted up twice (hmmm, actually, maybe there were two ClientStorages, you'd have 
to check with Shane, I'm not sure of the semantics of how ExternalMount works in this situation). 
That, apparently, causes a deadlock ;-)

What I now have, which works, is one storage server serving up three filestorages.
I still have two ExternalMount objects but they now connect using ClientStorage to two seperate
storages, which just happen to eb served from the same storage server...

Hmm... my head hurst now, can I go home? ;-)

cheers,

Chris

----------------------------------------------------------------------

Comment By: Jim Fulton (dcjim)
Date: 2002-04-30 07:27

Message:
Logged In: YES 
user_id=73023

Ignore my previous comment. I should have read Shane's
comment
more closely.

Shane,

Did Chris have two clients open on the same storage, or just
the
same storage server?  Chris said in email that using two
separate
storage servers didn't help.

Jim

----------------------------------------------------------------------

Comment By: Jim Fulton (dcjim)
Date: 2002-04-30 07:21

Message:
Logged In: YES 
user_id=73023

Opening multiple clients to the *same* underlying storage is
not
supported.


----------------------------------------------------------------------

Comment By: Shane Hathaway (hathawsh)
Date: 2002-04-29 16:38

Message:
Logged In: YES 
user_id=16701

I'm not sure ZEO should be involved in the solution.  Chris
had two client storages, pointing to the same storage,
mounted in different places in his ZODB.  At the ZEO level,
we'd have to revise the way locking works and make sure that
the different client storages don't write conflicting
changes to any object.  But if we instead revised the
database mounting code, we could avoid the need to use more
than one client storage to mount multiple parts of a
database.  There could be a central list of databases and a
list of mount points that refers to the list of databases. 
We could avoid opening any database more than once.

But we don't need to do all that right now since Chris found
a simpler solution: mount two databases instead of one.  And
database mounting is still an experimental feature.

In any case, perhaps we could find a way to turn the current
deadlock situation into an error.



----------------------------------------------------------------------

Comment By: Chris Withers (fresh)
Date: 2002-04-29 15:55

Message:
Logged In: YES 
user_id=24723

Well, is there any way to make it so that it doesn't matter whether or not ZEO knows that multiple clients 
are participating in the transaction? I mean, it strikes me that this deadlock should probably not occur, 
regardless of whether ZEO knows they're part of the same transaction or not...

...but of course, I know very little. Shane would be the man to discuss this with :-)

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2002-04-29 15:15

Message:
Logged In: YES 
user_id=31392

This could be really hard to fix...  I don't see a
principled way for a ZEO to know that multiple clients are
part of the same transaction.  Maybe the transaction object
can grow some metadata.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=115628&aid=550246&group_id=15628