Re: ZEO with Multiple Storages
One ZEO server can deal with multiple storages just fine. Transactions involving multiple storages commit just fine.
Caveat: You buy nothing by having a single ZEO process serve several ZODBs when you're trying to decrease commit times. You should run a ZEO process per ZODB you're serving out, that way the writes can be segregated and parallelized. That also makes it easier to move processes/databases around disks/hosts to spread I/O load.
Just to clarify, you are saying that a single ZEO server would not be able to deal with multiple storages simultaneously and that a transaction in one storage would block transactions in all other storages. However, if we were to run multiple ZEO processes this would not be the case? -Brian
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 14 Sep 2006, at 20:49, Brian Brinegar wrote:
One ZEO server can deal with multiple storages just fine. Transactions involving multiple storages commit just fine.
Caveat: You buy nothing by having a single ZEO process serve several ZODBs when you're trying to decrease commit times. You should run a ZEO process per ZODB you're serving out, that way the writes can be segregated and parallelized. That also makes it easier to move processes/databases around disks/hosts to spread I/O load.
Just to clarify, you are saying that a single ZEO server would not be able to deal with multiple storages simultaneously and that a transaction in one storage would block transactions in all other storages.
ZEO server processes are single threaded, access to the databases must be serialized.
However, if we were to run multiple ZEO processes this would not be the case?
Right. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (Darwin) iD8DBQFFCk3ARAx5nvEhZLIRAr4zAJ9IsO1Hv1t1QxOgP8Hoq5pTZwyQCACfRpEO fNmfNMHm2mhjhbTRPKt5pyY= =SXPM -----END PGP SIGNATURE-----
Brian Brinegar wrote at 2006-9-14 14:49 -0400:
One ZEO server can deal with multiple storages just fine. Transactions involving multiple storages commit just fine.
Caveat: You buy nothing by having a single ZEO process serve several ZODBs when you're trying to decrease commit times. You should run a ZEO process per ZODB you're serving out, that way the writes can be segregated and parallelized. That also makes it easier to move processes/databases around disks/hosts to spread I/O load.
Just to clarify, you are saying that a single ZEO server would not be able to deal with multiple storages simultaneously and that a transaction in one storage would block transactions in all other storages.
Jens might be saying this -- but it is not (completely) true: A transaction is not an atomic operation for the ZEO server. Therefore, other transactions can proceed. A ZODB transaction commit consists of the following stages: tpc_begin # the first transaction stage begins ... commit all modified object ... vote # first stage finishes tpc_finish # second stage finishes ZEO processes these stages as follows: tpc_begin -- a commit log is created ... commit ... -- the new state is stored in the commit log vote -- ZEO acquires a storage look -- no further commits for this storage possible storage "tpc_begin" called objects from commit log stored storage "vote" called tpc_finish -- storage "tpc_finish" called; storage lock released Jens is right that ZEO is (essentially) a single threaded server. When it processes a request other requests must wait until this request is finished. Using multiple ZEO servers will increase parallelism. However, a transaction usually consists of many requests to a ZEO server and between such requests ZEO is happy to perform other requests even those for other transaction commits. Only once "vote" started, a storage is "commit" blocked until the following "tcp_finish|abort". Other storages can be processed in a normal way; they can even commit transactions. By the way: for large transactions "vote" may take a long time (and prevent other requests from being processed). That's why I implemented "vote" in our (private) Zope copy as a "long running request". Such requests are run in a separate thread (ZEO is not really a single threaded server -- it is only that most commands are run in the main thread) and allow other requests to be processed in the meantime. -- Dieter
participants (3)
-
Brian Brinegar -
Dieter Maurer -
Jens Vagelpohl