ZEO cache instrumentation -- any takers?
A while ago I announced a new ZEO cache instrumentation feature, and asked if anyone was interested in enabling this instrumentation in their site. I got exactly zero responses... :-( I'd like to repeat the offer. From the instrumentation data, it is easy to determine the most effective cache size for your site (a tool to do this is part of the release). All you need to do is replace one file, ClientCache.py. Both ZEO 1 and ZEO 2 are supported. The instrumentation code does *not* use zLOG and causes very little overhead. It can create a large log file though (100 MB per day at one Zope Corp customer). Here's a copy of my original announement:
Subject: ZEO cache analysis tools prerelease From: Guido van Rossum <guido@python.org> To: zope-announce@zope.org Date: Tue, 10 Sep 2002 17:02:41 -0400
* Do you want to know if your ZEO client cache is configured properly? * Are you using ZEO but you didn't know it had a client cache? * Do you want to contribute towards a better ZEO client cache design? If you answered yes at least once, read on! Jeremy is working on a new ZEO 2 release; the first beta was released two weeks ago. For that release I have written cache instrumentation code that lets you review the effectiveness of the ZEO client cache. There's also a version of the instrumented cache for ZEO 1. I don't plan to work on it more before the ZEO 2 release (unless I receive bug reports). All code is available *now* from the ZODB3 CVS tree: http://cvs.zope.org/ZODB3/. In addition to adding instrumentation, some bugs affecting performance but not correctness of the cache have been fixed in the instrumented cache code. The instrumentation is controlled by an environment variable so you can leave the new cache code in place after turning off the instrumentation. I'm inviting everyone who is running ZEO seriously to install the cache instrumentation code, run it for a while, and send me the resulting trace file. (The trace files contain no privacy-sensitive data, so you don't have to worry about revealing anything about your site.) The trace files will be used as input to a simulation of a new cache design. The only way to validate a cache design is to run experiments, and trace files are the most reliable way to run such experiments. The more trace files I receive the better. The larger they are the better (traces of several days of uninterrupted service are ideal). I've got the disk space to burn. I should mention that I'm confident that the instrumentated cache works correctly: we've been collecting traces without problems at a Zope Corp customer site since last Wednesday. To enable the cache instrumentation, you need to replace a single file in the ZEO package used by the Zope application, set an environment variable, and restart the Zope application. The file you need to replace is named ClientCache.py. Here are the URLs: For ZEO 1: http://cvs.zope.org/ZODB3/ZEO1/ClientCache.py?rev=HEAD For ZEO 2: http://cvs.zope.org/ZODB3/ZEO/ClientCache.py?rev=HEAD Make sure to get the file for the ZEO version you are using! Details about enabling the tracing can be found here: http://cvs.zope.org/ZODB3/ZEO/README.txt?rev=HEAD If you want to play with the statistics or simulation programs yourself, get stats.py and simul.py from the ZEO directory: http://cvs.zope.org/ZODB3/ZEO/ These work with trace files produced by either version of the instrumented cache code. Once you have one or more trace files for me, the best way to get it to me is probably to place a gzipped copy on an FTP or HTTP server from which I can download it and mail me the URL. As I said, these trace files contain no sensitive data. If you still prefer a different approach, email me and we'll figure something out. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Wed, 2002-10-02 at 16:14, Guido van Rossum wrote:
A while ago I announced a new ZEO cache instrumentation feature, and asked if anyone was interested in enabling this instrumentation in their site. I got exactly zero responses... :-(
I'd like to repeat the offer.
We (Hiperlógica) have some ZEO installations: * A single ZEO client with UNIX socket that was used because Zope used to crash a lot before Python 2.1.3 and restarting was way faster with ZEO (hi Matt!). This one is in production w/ 75k hits/day. * A single ZEO client with a ZEO server in a different machine, both Win2k (the client intends to add more ZEO clients and a load balancer (eventually) but insisted on testing this otherwise meaningless configuration) * Two identical Win2k machines running a ZEO client and a ZEO server each. Both ZEO clients connect to the ZEO server on the first machine, but the ZEO server on the second machine is there for manual failover. Following Guido's suggestion, we will be installing that instrumentation in all of those sites but since we're talking about ZEO, lemme ask for some help: The last two installations belong to the same client, and are both experiencing a problem that I believe is related to ZEO latency. Both sites depend heavily on a single ZCatalog for operation (one ZCatalog hit in almost every page, with some pages with more than one ZCatalog hit). When site content is being managed, we're hitting ReadConflict errors (They read like this: ZODB.POSException.ReadConflictError on database read conflict error (oid000000000004c0a4, class BTrees.OOBTree.OOBucket)). The reference to BTrees makes me think this is ZCatalog related, we don't use BTrees ourselves anywhere else and we only use regular Folders as bases for our folderish objects, and managing content causes the reindexing of objects. Which I think is what is triggering the conflict errors. You guys have any tips to avoid this or to help narrow this problem down? -- Ideas don't stay in some minds very long because they don't like solitary confinement.
"LRA" == Leonardo Rochael Almeida <leo@hiper.com.br> writes:
LRA> The last two installations belong to the same client, and are LRA> both experiencing a problem that I believe is related to ZEO LRA> latency. Both sites depend heavily on a single ZCatalog for LRA> operation (one ZCatalog hit in almost every page, with some LRA> pages with more than one ZCatalog hit). When site content is LRA> being managed, we're hitting ReadConflict errors (They read LRA> like this: ZODB.POSException.ReadConflictError on database read LRA> conflict error (oid000000000004c0a4, class LRA> BTrees.OOBTree.OOBucket)). The reference to BTrees makes me LRA> think this is ZCatalog related, we don't use BTrees ourselves LRA> anywhere else and we only use regular Folders as bases for our LRA> folderish objects, and managing content causes the reindexing LRA> of objects. Which I think is what is triggering the conflict LRA> errors. Your analysis sounds correct. A read conflict occurs when on thread modifies an object while another thread is reading it (subject to certain other conditions that probably aren't relevant). I assume several users are managing content at the same time. If Zope gets a read conflict, it will retry the transaction. So you must get very unlikely in order to see the error. Each time the transaction is retried, something must go wrong. Are there a lot of concurrent transactions? Or do the updates affect a large number of objects at once? If either is true, it seems like it would make the problem more likely to occur. Jeremy
On Thu, 2002-10-03 at 01:11, Jeremy Hylton wrote:
"LRA" == Leonardo Rochael Almeida <leo@hiper.com.br> writes:
LRA> The last two installations belong to the same client, and are LRA> both experiencing a problem that I believe is related to ZEO LRA> latency. Both sites depend heavily on a single ZCatalog for LRA> operation (one ZCatalog hit in almost every page, with some LRA> pages with more than one ZCatalog hit). When site content is LRA> being managed, we're hitting ReadConflict errors (They read LRA> like this: ZODB.POSException.ReadConflictError on database read LRA> conflict error (oid000000000004c0a4, class LRA> BTrees.OOBTree.OOBucket)). The reference to BTrees makes me LRA> think this is ZCatalog related, we don't use BTrees ourselves LRA> anywhere else and we only use regular Folders as bases for our LRA> folderish objects, and managing content causes the reindexing LRA> of objects. Which I think is what is triggering the conflict LRA> errors.
Your analysis sounds correct. A read conflict occurs when on thread modifies an object while another thread is reading it (subject to certain other conditions that probably aren't relevant). I assume several users are managing content at the same time.
Not really. The system is in the last stages of functional testing with the client (it'd be funny if it wasn't so tragic :-), so we have a couple of insertions and modifications happening at the same time the public interface is being viewed.
If Zope gets a read conflict, it will retry the transaction. So you must get very unlikely in order to see the error. Each time the transaction is retried, something must go wrong.
That's the assumption I was working under.
Are there a lot of concurrent transactions?
Not really, as I said, we're only testing.
Or do the updates affect a large number of objects at once?
Not really, unless the ZCatalog update changes a bunch of BTree subobjects sequentially. Even then, the sleep between retries should take care of that, unless the ZEO latency is really high, which I don't think is enough to explain.
If either is true, it seems like it would make the problem more likely to occur.
True, but the problem could be something else entirely: We setup the standard error page to call an external method that writes the exception data (Exception, request path and traceback) to zLOG. One thing I just noticed is this little snippet in the stupid log:
2002-09-25T19:57:36 ERROR(200) FieldIndex unindex_object could not remove documentId 183870929 from index status. This should not happen. Traceback (innermost last): File E:\aplic\ZCLIEN~1\lib\python\Products\PluginIndexes\common\UnIndex.py, line 168, in removeForwardIndexEntry (Object: status) File E:\aplic\ZCLIEN~1\lib\python\ZODB\Connection.py, line 463, in setstate ReadConflictError: database read conflict error (oid 000000000005cffa, class BTrees.IIBTree.IITreeSet) <<< It seems unindex_object is indiscriminately trapping ReadConflictErrors instead of letting ZPublisher deal with it. I believe this is not the cause of us seeing ReadConflictErrors in the browser, but I decided to point this out since it looks like a bug to me. The message above was caused by an 'exceptionless' (that a word?) `exception:' clause in lib/python/Products/PluginIndexes/common/UnIndex.py:178 And then, the lightning bolt of illumination hits me... I see ZPublisher generated conflict errors in the stupid log that aren't followed by the log messages generated by the standard_error_page, but the conflict errors that ARE trapped by standard_error_page aren't followed by ZPublisher 'conflict error' messages. So I go searching and what do I find?
lib/python/Products/PageTemplates/TALES.py:253+ [... def evaluate(self, expression ...] except: raise TALESError, (`expression`, sys.exc_info(), self.position), sys.exc_info()[2] else: return v <<<
So ZPublisher isn't retrying this ConflictError because it isn't getting any! it's getting a TALESError! which it's dutifully reporting via the usual mechanisms! Well, this is an obvious error, but I'm too tired now to think of a fix. Besides, in a few minuts I have to watch the debate between the presidential candidates for the election this Sunday. Cheers, Leo PS: Hey, Guido, any chance of deprecating exceptionless 'exception:' clauses anytime soon? We got a bunch of those making our life harder on Zope :-) -- Ideas don't stay in some minds very long because they don't like solitary confinement.
PS: Hey, Guido, any chance of deprecating exceptionless 'exception:' clauses anytime soon? We got a bunch of those making our life harder on Zope :-)
No chance for backwards compatibility, but I try to stamp them out whenever I see them. Submit a tracker issue about the ones that particularly bug you -- with patch if you've got a suggestion. We'll get there still. --Guido van Rossum (home page: http://www.python.org/~guido/)
As much as I try to avoid them (especially in Zope code), they are sometimes necessary because you simply don't know what exceptions might be raised from inside Python or the standard libs. Besides, even if you stamped it out people will just use: try: ... except Exception: Besides, sometimes you really do want to trap all exceptions, do something and then re-raise them again. IMO that's not bad form if its not habitual. Even simple operations in Python itself can raise various exceptions. For instance, a humble int(x) can raise TypeError or ValueError, and Guido knows what else ;^). That leads to me being less confident in my exception handling then I would like to be. At any rate Chris McDonough and I recently had a conversation about ZPT catching all exceptions and his sneeking suspicion that it was a bad thing with regard to read conflicts, but I think we concluded it wasn't as bad as he thought it might be. Maybe we were wrong. Since you have identified these places in the code, I think it would be worthwhile to add the following above the bare excepts in question to see what happens: except ReadConflictError: raise If nothing else, you could rule this out. -Casey On Thursday 03 October 2002 08:58 pm, Guido van Rossum wrote:
PS: Hey, Guido, any chance of deprecating exceptionless 'exception:' clauses anytime soon? We got a bunch of those making our life harder on Zope :-)
No chance for backwards compatibility, but I try to stamp them out whenever I see them. Submit a tracker issue about the ones that particularly bug you -- with patch if you've got a suggestion.
We'll get there still.
--Guido van Rossum (home page: http://www.python.org/~guido/)
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
except ReadConflictError: raise
Or even better (since ReadConflictError inherits from ConflictError): except ConflictError: raise What would be nice is a way to define in Python a kind of exception that is not caught by bare "except:" statements but only by "except SpecificClass:" statements. Not quite an uncatchable exception, but one that is caught only by except statements that name it. - C
What would be nice is a way to define in Python a kind of exception that is not caught by bare "except:" statements but only by "except SpecificClass:" statements. Not quite an uncatchable exception, but one that is caught only by except statements that name it.
I'm skeptical about this one. Can you explain the use case? You can probably fake this to a large extent by using "except Exception:" instead of "except:" everywhere; then you can raise an exception that does not inherit from Exception. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Fri, 2002-10-04 at 08:21, Guido van Rossum wrote:
What would be nice is a way to define in Python a kind of exception that is not caught by bare "except:" statements but only by "except SpecificClass:" statements. Not quite an uncatchable exception, but one that is caught only by except statements that name it.
I'm skeptical about this one. Can you explain the use case?
I am too. ;-) Zope (for better or worse) uses the same machinery that users use in scripts and templates to deal with exceptional conditions (ConflictError, Unauthorized, etc). It should not be possible for users to catch these kinds of errors in Python Scripts and/or TAL. The two "exception domains" should really be divorced in some way. How about "goto"? ;-) Just kidding.
You can probably fake this to a large extent by using "except Exception:" instead of "except:" everywhere; then you can raise an exception that does not inherit from Exception.
This is probably the thing to do. - C
As much as I try to avoid them (especially in Zope code), they are sometimes necessary because you simply don't know what exceptions might be raised from inside Python or the standard libs.
Sure. But the point is that *historically* Zope code (and lots of other Python code!) has contained a lot of places where "except:" was used when the author knew exactly which exceptions he was expecting. To make it worse when this is sometimes done for a large stretch of code, even though there are only a few specific spots where the exception is expected. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Friday 04 Oct 2002 1:18 pm, Guido van Rossum wrote:
To make it worse when this is sometimes done for a large stretch of code, even though there are only a few specific spots where the exception is expected.
The idiom of putting this long stretch of code into and 'else' block after the 'try' block is underused, and not just in Zope. I wonder why?
Toby Dickenson wrote:
On Friday 04 Oct 2002 1:18 pm, Guido van Rossum wrote:
To make it worse when this is sometimes done for a large stretch of code, even though there are only a few specific spots where the exception is expected.
The idiom of putting this long stretch of code into and 'else' block after the 'try' block is underused, and not just in Zope. I wonder why?
I think it's because its syntax is out of order. :-) Whenever I'm about to write a "try...except...else" block, I really want to spell it "try...else...except". This places the exceptional situation after the common situation, where it should be. Unfortunately the word "else" has too narrow a meaning to allow this, so I wouldn't recommend it actually be done. :-) Shane
Shane Hathaway <shane@zope.com> wrote:
The idiom of putting this long stretch of code into and 'else' block after the 'try' block is underused, and not just in Zope. I wonder why?
I think it's because its syntax is out of order. :-) Whenever I'm about to write a "try...except...else" block, I really want to spell it "try...else...except". This places the exceptional situation after the common situation, where it should be.
Yep, that's exactly what I was about to say. (Not that I suggest changing the syntax either.) Florent -- Florent Guillaume, Nuxeo (Paris, France) +33 1 40 33 79 87 http://nuxeo.com mailto:fg@nuxeo.com
On Sat, 5 Oct 2002, Shane Hathaway wrote:
I think it's because its syntax is out of order. :-) Whenever I'm about to write a "try...except...else" block, I really want to spell it "try...else...except". This places the exceptional situation after the common situation, where it should be. Unfortunately the word "else" has too narrow a meaning to allow this, so I wouldn't recommend it actually be done. :-)
I think you are right. I always have to look up 'else' to make sure I'm using it right, because it isn't completely intuitive. I can never remember if 'else' means "if we had an exception, do this" (try...else) or "if we didn't have an exception, do this" (except...else). Actually, having written that out I'll probably remember it now <grin>. How about "try...then...except"? --RDM
On Fri, 2002-10-04 at 00:13, Casey Duncan wrote:
As much as I try to avoid them (especially in Zope code), they are sometimes necessary because you simply don't know what exceptions might be raised from inside Python or the standard libs. Besides, even if you stamped it out people will just use:
try: ... except Exception:
in which case we could do as Guido suggested and raise something that doesn't inherit from Exception :-)
Besides, sometimes you really do want to trap all exceptions, do something and then re-raise them again. IMO that's not bad form if its not habitual.
Or the ZPublisher case where the error is dealt with by rolling back transactions, reporting and proceeding as if nothing had happened. Nobody wants their web app dying just because they didn't think of a ZeroDivisionError in a PythonScript :-) But still, those are clear cut cases and the error is not simply ignored and swept under the carpet.
Even simple operations in Python itself can raise various exceptions. For instance, a humble int(x) can raise TypeError or ValueError, and Guido knows what else ;^). That leads to me being less confident in my exception handling then I would like to be.
I understand the feeling :-)
At any rate Chris McDonough and I recently had a conversation about ZPT catching all exceptions and his sneeking suspicion that it was a bad thing with regard to read conflicts, but I think we concluded it wasn't as bad as he thought it might be. Maybe we were wrong.
Well, now we know. It's a bad thing because the requests aren't being retried at all, and the web app is failing in a very visible way (to the astonished client) which could be avoided if the exception wasn't trapped. It's not as bad as it could be, since the code ends up raising another exception and so the transaction is rolled back and no damage to the ZODB is done, but compare that with the code in lib/python/Products/PluginIndexes/common/UnIndex.py:178: except: LOG(self.__class__.__name__, ERROR, ('unindex_object could not remove ' 'documentId %s from index %s. This ' 'should not happen.' % (str(documentId), str(self.id))), '', sys.exc_info()) This code is trapping read- (and possibly write-) conflict errors, logging them and proceeding blindly. This could as well be causing the silent ZCatalog corruptions that were discussed a few days ago! I find it slightly disturbing that a code that logs 'This should not happen' just lets the thing that shouldn't be happening go on, and almost doesn't make any fuss about it! :-)
Since you have identified these places in the code, I think it would be worthwhile to add the following above the bare excepts in question to see what happens:
except ReadConflictError: raise
If nothing else, you could rule this out.
This will certainly solve the problem for the TALES code (with ConflictError as Chris sugested (or maybe TransactionError or POSError :-)) but the UnIndex Code looks really b0rked to me. I'm afraid that leaving a bare exception there (even excluding the ConflictErrors) will give us trouble in the future (we are very ZCatalog dependant here), but I don't know enough about the ZCatalog to know what removing it would entail. I'm also afraid that if I file an issue on it, it will lie there in the collector forever because no one understands why a bare except was left there... Back in the TALES case, I understand the unconditional capturing of exceptions is done so that another exception can be raised that informs the user about the line and column (and expression) in ZPT where the error ocurred, but I don't think that masking the exception with another exception is the correct way for this. I think the real solution would be a way to manipulate the traceback so that we can insert meaningful information about languages we are interpreting without actually losing the original expression and it's traceback. I picture something along the lines: except: reraise "TALES error on template %s: line %d column %d" % \ (expression.template_path, expression.template_line, expression.template_column) So that this information would be displayed along the right frame of the traceback instead of replacing it. Maybe it wouldn't even need another reserved word, a builtin function reraise() or a sys.reraise() could be enough. I'm testing a fix for the TALES case along the lines of what Casey sugested and will report back with results. Cheers, Leo -- Ideas don't stay in some minds very long because they don't like solitary confinement.
On Fri, 2002-10-04 at 16:37, Leonardo Rochael Almeida wrote:
I'm testing a fix for the TALES case along the lines of what Casey sugested and will report back with results.
And here it is. TALES actually had a slot for my change, go figure :-) as for PluginIndex/common/UnIndex.py, I'd like to propose the following rule, before I attempt a fix: No bare 'except:' shall silently ingore it's exception and proceed. Outside of ZPublisher, any bare 'except:' MUST raise either the original exception or another one. Inside of ZPublisher it's too dark to read. What do you think? Cheers, Leo PS: in PageTemplates/TALES.py there's another bare 'except:' in the Iterator class, I suggest it be changed to include a self._nocatch mechanism just like the Context class in that same file. What do you think? -- Ideas don't stay in some minds very long because they don't like solitary confinement. --- lib/python/Products/PageTemplates/Expressions.py-orig 2002-10-04 17:26:31.000000000 +0000 +++ lib/python/Products/PageTemplates/Expressions.py 2002-10-04 17:26:38.000000000 +0000 @@ -24,6 +24,7 @@ TALESError, Undefined, Default, _parse_expr from string import strip, split, join, replace, lstrip from Acquisition import aq_base, aq_inner, aq_parent +from ZODB.POSException import ConflictError _engine = None @@ -33,7 +34,7 @@ from PathIterator import Iterator _engine = Engine(Iterator) installHandlers(_engine) - _engine._nocatch = (TALESError, 'Redirect') + _engine._nocatch = (TALESError, 'Redirect', ConflictError) return _engine def installHandlers(engine):
On Fri, 2002-10-04 at 17:58, Leonardo Rochael Almeida wrote:
On Fri, 2002-10-04 at 16:37, Leonardo Rochael Almeida wrote:
I'm testing a fix for the TALES case along the lines of what Casey sugested and will report back with results.
And here it is. [...]
BTW, the _nocatch that I patched had literal string 'Redirect' in it. Now considering that exception catching is done by the 'is' operator (or by isinstance(), in case of object instances) isn't it possible, or even likely that the 'except self._nocatch:' in PageTemplates/TALES.py wouldn't be able to catch other 'Redirect's?
--- lib/python/Products/PageTemplates/Expressions.py-orig 2002-10-04 17:26:31.000000000 +0000 +++ lib/python/Products/PageTemplates/Expressions.py 2002-10-04 17:26:38.000000000 +0000 @@ -24,6 +24,7 @@ TALESError, Undefined, Default, _parse_expr from string import strip, split, join, replace, lstrip from Acquisition import aq_base, aq_inner, aq_parent +from ZODB.POSException import ConflictError
_engine = None @@ -33,7 +34,7 @@ from PathIterator import Iterator _engine = Engine(Iterator) installHandlers(_engine) - _engine._nocatch = (TALESError, 'Redirect') + _engine._nocatch = (TALESError, 'Redirect', ConflictError) return _engine
def installHandlers(engine): -- Ideas don't stay in some minds very long because they don't like solitary confinement.
Leonardo Rochael Almeida wrote:
I'm testing a fix for the TALES case along the lines of what Casey sugested and will report back with results.
Leo, This is a good thing to work on but you really should work on the trunk. The TALES exception handling was redone after Zope 2.5. Exceptions don't get transformed into TALESErrors any more. The line and column number appear in custom-formatted tracebacks. The _nocatch mechanism is brittle, but maybe it's all we can get for now. This problem of catching all errors, including ConflictErrors, is quite deep, since even untrusted Python scripts are allowed to do this. Here's an idea: if a conflict (read or write) occurs in a connection, that connection turns read-only until the next transaction boundary. If the application tries to write using it, ZODB raises another ConflictError. I wonder if that policy would work. Shane
Guido van Rossum wrote:
A while ago I announced a new ZEO cache instrumentation feature, and asked if anyone was interested in enabling this instrumentation in their site. I got exactly zero responses... :-(
Hmmm... people might have done it and not let you know. Are you looking to get the log files back so you can analyse them? And can you just put the new ClientCache.py on one ZEO client in the configuration? cheers, Chris
Guido van Rossum wrote:
A while ago I announced a new ZEO cache instrumentation feature, and asked if anyone was interested in enabling this instrumentation in their site. I got exactly zero responses... :-(
Hmmm... people might have done it and not let you know. Are you looking to get the log files back so you can analyse them?
Yes, that was what I was explicitly asking if you read back the announcement.
And can you just put the new ClientCache.py on one ZEO client in the configuration?
Yes. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
A while ago I announced a new ZEO cache instrumentation feature, and asked if anyone was interested in enabling this instrumentation in their site. I got exactly zero responses... :-(
I'd like to repeat the offer. From the instrumentation data, it is easy to determine the most effective cache size for your site (a tool to do this is part of the release). All you need to do is replace one file, ClientCache.py. Both ZEO 1 and ZEO 2 are supported. The instrumentation code does *not* use zLOG and causes very little overhead. It can create a large log file though (100 MB per day at one Zope Corp customer).
I just tried enabling the logging, shortly afterwards, I got: 2002-10-04T16:05:44 PANIC(300) Traceback (innermost last): Module __main__, line 94, in ? Module ZODB.Transaction, line 161, in commit Module ZODB.Transaction, line 222, in commit Module ZODB.Transaction, line 195, in commit Module ZODB.Transaction, line 256, in _commit_objects Module ZODB.Connection, line 387, in commit - __traceback_info__: (('BTrees._IOBTree', 'IOBucket'), '\x00\x00\x00\x00\x00 \x06\xd6\xc3', '') InvalidObjectReference: Attempt to store an object from a foreign database conne ction Not sure it's related, but I've never had that happen before :-S cheers, Chris
Have you tried clearing your cache after reverting the ClientCache.py change? Are you sure you didn't use the ZEO2 ClientCache.py with a ZEO1 installation or vice versa? --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
Have you tried clearing your cache after reverting the ClientCache.py change?
Where would I find this cache? How would I clear it?
Are you sure you didn't use the ZEO2 ClientCache.py with a ZEO1 installation or vice versa?
Not 100%, but I don't think I did... cheers, Chris
participants (10)
-
Casey Duncan -
Chris McDonough -
Chris Withers -
Florent Guillaume -
Guido van Rossum -
jeremy＠zope.com -
Leonardo Rochael Almeida -
R. David Murray -
Shane Hathaway -
Toby Dickenson