We've just released ZEO 2.0 beta 1.It's available from http://www.zope.org/Products/ZEO. We think this release of ZEO 2 is fairly stable, although we may do one more beta release to add some performance improvements. There are many changes since ZEO 2.0a1, which was released in May. The CHANGES.txt file describes the most significant changes; I've included the 2.0b1 changes list at the end of this message. The ZEO 2 protocol is not backwards compatible with ZEO 1. If you want to upgrade a system, all the clients and servers need to be upgraded at the same time. I don't plan to provide backwards compatibility for 2.0, but could be convinced otherwise if there is significant user demand. In the alpha release, we reported that the test suite was a bit fragile on Windows. It is much better now. We have tested on Windows 2000 and Windows 98, and expect all the tests to pass. The tests may run very slow on Windows, because many of the tests spawn server processes. The Windows scheduler can make this very slow. On occasion, we've seen the tests hang for several minutes before finishing successfully. After the alpha release, I suggested that there would be several more alpha releases to add new features. We didn't have the resources to pursue the new features. Instead, we plan to get a solid 2.0 final release out as soon as possible. New features will be postponed until 2.1. Jeremy The changes since ZEO 2.0a1 release are: - Added version identification. The variable ZEO.version is a string identifying the version in the style of sys.version. The same string is also available in ZEO/version.txt for people who want to check without loading the code. - Fixed performance problem. The low-level socket code was often calling send() with very small strings instead of joining them together before calling send. The small string variant ran afoul of the Nagle algorithm and delayed TCP acks. - Fix bugs in server handling of queued transactions Several bugs in 2.0a1 could cause the storage to lock up when it attempted to restart a transaction that was queued waiting for an earlier transaction to finish. The locking code in the two-phase commit methods was revised to properly use the condition variable protocol. Thanks to Tim Peters for pointing this out. - Avoid locking up server on unexpected client exit If a client disconnected unexpectedly in the middle of a two- phase commit, the server would never release the commit lock. - Very long log messages avoided. Some of the logging messages at BLATHER and lower were producing very long log messages because they included all the argument of method calls, even when the argument was a list with thousands of elements. These log messages are now truncated at a reasonable length. - Fix log rotation The SIGHUP behavior was fixed. If the ZEO server receives a SIGHUP, it will re-initialize its logging code. Thus, it is possible to rotate the log files and then send a HUP. - Fixed problem handling signals. The asyncore and zrpc error handlers would trap SystemExit if it was raised by a signal handler. They now ignore it. - pack() can be called on a read-only connection This ends up being very convenient for packing a read-only server, even though it isn't obvious that pack() is a read- only operation. - pack(wait=1) now waits A bug caused pack(wait=1) to return immediately, even though the pack was still running. - Fix bug in history(). The history() implementation prevent the client from returning more than one item. - Enhanced logging for blocked transactions. If a transaction had to be blocked because another transaction was executing, the server would log an info message. Now it also logs a message when the transaction is resumed, so that it doesn't look like the transaction was blocked forever. - Fixed bug in logging call for store(). The logging call could raise NameError. - Removed debug argument to ClientStorage constructor. It had no effect. - Removed ZEO.trigger There were two copies of the trigger module in the last release. The one in ZEO.trigger was removed, because it is only needed in ZEO.zrpc.trigger. - Added minimal tests of start.py. There are only run on Unix.
On Friday 23 Aug 2002 3:59 pm, Jeremy Hylton wrote:
We've just released ZEO 2.0 beta 1.It's available from http://www.zope.org/Products/ZEO.
We think this release of ZEO 2 is fairly stable,
hurrah!
although we may do one more beta release to add some performance improvements.
Do you have something specific in mind?
After the alpha release, I suggested that there would be several more alpha releases to add new features. We didn't have the resources to pursue the new features. Instead, we plan to get a solid 2.0 final release out as soon as possible. New features will be postponed until 2.1.
I would like to merge the branch 'toby-signal-branch' in before 2.0. This provides an easy way to use extensions to the storage API across ZEO. This feature is critical for DirectoryStorage. I am 100% happy with the current implementation (but not very happy with the name of the new method.... suggestions welcome) Any objections?
"TD" == Toby Dickenson <tdickenson@geminidataloggers.com> writes:
although we may do one more beta release to add some performance improvements.
TD> Do you have something specific in mind? I'd like to change this bit of code in handle_write(): if n < len(v): # XXX It's unfortunate that we end up making many # slices of a large string. output.insert(0, v[n:]) break # we can't write any more If we send a very large message, which could be caused by a big pickle or just a transaction that touches a lot of objects, the message gets sent in little chunks. The current implementation uses string slicing to save the rest of the string, but that ends up making many copies of the large string -- an O(n^2) proposition. A possible solution is to store an index into the first string in __output and just increment the index. The logic will be a bit tricky to get right.
After the alpha release, I suggested that there would be several more alpha releases to add new features. We didn't have the resources to pursue the new features. Instead, we plan to get a solid 2.0 final release out as soon as possible. New features will be postponed until 2.1.
TD> I would like to merge the branch 'toby-signal-branch' in before TD> 2.0. This provides an easy way to use extensions to the storage TD> API across ZEO. TD> This feature is critical for DirectoryStorage. I am 100% happy TD> with the current implementation (but not very happy with the TD> name of the new method.... suggestions welcome) I'll look at the branch before the next beta. Anyone have ideas for a new name? Jeremy
On Wednesday 28 Aug 2002 2:19 pm, Jeremy Hylton wrote:
I'd like to change this bit of code in handle_write():
if n < len(v): # XXX It's unfortunate that we end up making many # slices of a large string. output.insert(0, v[n:]) break # we can't write any more
If we send a very large message, which could be caused by a big pickle or just a transaction that touches a lot of objects, the message gets sent in little chunks. The current implementation uses string slicing to save the rest of the string, but that ends up making many copies of the large string -- an O(n^2) proposition.
A possible solution is to store an index into the first string in __output and just increment the index. The logic will be a bit tricky to get right.
There is an O(n) option with easy logic... fragment big strings inside message_output. diff attached. [hmmm. Its use of a python list as a fifo queue is O(n2) worst case, but I doubt we will be affected by that in practice]
"TD" == Toby Dickenson <tdickenson@geminidataloggers.com> writes:
TD> There is an O(n) option with easy logic... fragment big strings TD> inside message_output. Sounds like a possibility. It would still be nice to have something in handle_write() to avoid extra slicing on partial send()s. But your patch is nice and simple. Jeremy
participants (3)
-
Jeremy Hylton -
jeremy＠zope.com
-
Toby Dickenson