SESSION loosing data when system has multiple users
We have been using Zope 2.7.0-beta2 and we have been having a bit of trouble with SESSION loosing data when we have more than one user on the system. These problems usually show up as KeyErrors when a particular data element in SESSION cannot be found. If I set the zserver threads to 1 in zope.conf, the problem seems to go away (but the system isn't really usable either). So I assumed it was a thread-safety problem in the SESSION code. I decided to be brave and take a look at the code. To my surprize, I found a test file called stresstestMultiThread.py in the lib/python/Products/Sessions/stresstests directory that seemed to be the exact thing I wanted to test. It wouldn't run without a couple of minor modifications, but once I got it running, with more than three or four threads, the tests fail (never complete). Is this test file still valid? It looks like it hasn't been modified for almost a year now. If it is valid, has anyone looked into getting the tests to pass. I will probably look into it more myself, but before I do, I just want to make sure I'm not headed down a dead-end street. Thanks, Steve P.S. - The tests failed in both the 2.7.0b2-src and the what is currently in cvs.
What OS? That test was written obviously to make sure the system works... others have complained about similar issues. I'd like to be able to track this down, but I can't make the test fail under Red Hat 7.3. On Fri, 2003-10-17 at 15:56, Steve Jibson wrote:
We have been using Zope 2.7.0-beta2 and we have been having a bit of trouble with SESSION loosing data when we have more than one user on the system. These problems usually show up as KeyErrors when a particular data element in SESSION cannot be found. If I set the zserver threads to 1 in zope.conf, the problem seems to go away (but the system isn't really usable either). So I assumed it was a thread-safety problem in the SESSION code.
I decided to be brave and take a look at the code. To my surprize, I found a test file called stresstestMultiThread.py in the lib/python/Products/Sessions/stresstests directory that seemed to be the exact thing I wanted to test. It wouldn't run without a couple of minor modifications, but once I got it running, with more than three or four threads, the tests fail (never complete).
Is this test file still valid? It looks like it hasn't been modified for almost a year now. If it is valid, has anyone looked into getting the tests to pass. I will probably look into it more myself, but before I do, I just want to make sure I'm not headed down a dead-end street.
Thanks,
Steve
P.S. - The tests failed in both the 2.7.0b2-src and the what is currently in cvs.
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Chris, Sorry, I should have included more info, but my message was already a bit long. I have run the tests on both Red Hat 7.3 and Red Hat 9 with the same result. I am using python 2.2.3. Here is the step-by-step for what I did (I'm going to err on the side of too much info here): - check out latest Zope from cvs - cd Zope - ./configure - make - make instance - cd lib/python/Products/Sessions/stresstests - edit stresstestMultiThread.py - comment out line 38 - changed readiters and writeiters fro 100 to 10 on lines 117 and 118 - cp ~/Zope/lib/python/Products/Transience/tests/fauxtime.py . - python2 stresstestMultiThread.py This time, on Red Hat 7.3, I actually got an exception. I'll include the output below. Please let me know if there is ANYTHING I can do to help out with this one. I appreciate you looking into it. Steve ------------------------------------------------------------------------ [stevej@elwood stresstests]$ python2 stresstestMultiThread.py testNonOverlappingBrowserIds (__main__.TestMultiThread) ... 4 values in toc waiting for 8 threads read conflict 4 values in toc read conflict read conflict read conflict waiting for 8 threads read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict 4 values in toc 4 values in toc 4 values in toc 4 values in toc 4 values in toc 4 values in toc 4 values in toc 4 values in toc read conflict read conflict read conflict read conflict read conflict read conflict waiting for 8 threads read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict waiting for 8 threads read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict 0 read conflict read conflict read conflict read conflict read conflict waiting for 7 threads read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict 2003-10-17T14:41:56 25 Exception in thread Thread-3: Traceback (most recent call last): File "//usr/lib/python2.2/threading.py", line 414, in __bootstrap self.run() File "stresstestMultiThread.py", line 164, in run self.run1() File "stresstestMultiThread.py", line 187, in run1 data = session_data_manager.getSessionData(create=1) File "/home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py", line 93, in getSessionData return self._getSessionDataObject(key) File "/home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py", line 180, in _getSessionDataObject ob = container.new_or_existing(key) File "/home/stevej/Zope/lib/python/Products/Transience/Transience.py", line 175, in new_or_existing item = self.get(key, notfound) File "/home/stevej/Zope/lib/python/Products/Transience/Transience.py", line 807, in get v = self._data[b].get(k, notfound) KeyError: 0 read conflict read conflict read conflict 2003-10-17T14:41:56 17 Exception in thread Thread-2: Traceback (most recent call last): File "//usr/lib/python2.2/threading.py", line 414, in __bootstrap self.run() File "stresstestMultiThread.py", line 164, in run self.run1() File "stresstestMultiThread.py", line 187, in run1 data = session_data_manager.getSessionData(create=1) File "/home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py", line 93, in getSessionData return self._getSessionDataObject(key) File "/home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py", line 180, in _getSessionDataObject ob = container.new_or_existing(key) File "/home/stevej/Zope/lib/python/Products/Transience/Transience.py", line 175, in new_or_existing item = self.get(key, notfound) File "/home/stevej/Zope/lib/python/Products/Transience/Transience.py", line 807, in get v = self._data[b].get(k, notfound) File "/home/stevej/Zope/lib/python/ZODB/Connection.py", line 561, in setstate p, serial = self._storage.load(oid, self._version) File "/home/stevej/Zope/lib/python/tempstorage/TemporaryStorage.py", line 94, in load s=self._index[oid] KeyError: 2003-10-17T14:41:56 18 Exception in thread Thread-4: Traceback (most recent call last): File "//usr/lib/python2.2/threading.py", line 414, in __bootstrap self.run() File "stresstestMultiThread.py", line 164, in run self.run1() File "stresstestMultiThread.py", line 187, in run1 data = session_data_manager.getSessionData(create=1) File "/home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py", line 93, in getSessionData return self._getSessionDataObject(key) File "/home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py", line 180, in _getSessionDataObject ob = container.new_or_existing(key) File "/home/stevej/Zope/lib/python/Products/Transience/Transience.py", line 175, in new_or_existing item = self.get(key, notfound) File "/home/stevej/Zope/lib/python/Products/Transience/Transience.py", line 807, in get v = self._data[b].get(k, notfound) File "/home/stevej/Zope/lib/python/ZODB/Connection.py", line 561, in setstate p, serial = self._storage.load(oid, self._version) File "/home/stevej/Zope/lib/python/tempstorage/TemporaryStorage.py", line 94, in load s=self._index[oid] KeyError: read conflict read conflict read conflict read conflict read conflict waiting for 4 threads read conflict read conflict read conflict read conflict read conflict read conflict Chris McDonough wrote:
What OS? That test was written obviously to make sure the system works... others have complained about similar issues. I'd like to be able to track this down, but I can't make the test fail under Red Hat 7.3.
On Fri, 2003-10-17 at 15:56, Steve Jibson wrote:
We have been using Zope 2.7.0-beta2 and we have been having a bit of trouble with SESSION loosing data when we have more than one user on the system. These problems usually show up as KeyErrors when a particular data element in SESSION cannot be found. If I set the zserver threads to 1 in zope.conf, the problem seems to go away (but the system isn't really usable either). So I assumed it was a thread-safety problem in the SESSION code.
I decided to be brave and take a look at the code. To my surprize, I found a test file called stresstestMultiThread.py in the lib/python/Products/Sessions/stresstests directory that seemed to be the exact thing I wanted to test. It wouldn't run without a couple of minor modifications, but once I got it running, with more than three or four threads, the tests fail (never complete).
Is this test file still valid? It looks like it hasn't been modified for almost a year now. If it is valid, has anyone looked into getting the tests to pass. I will probably look into it more myself, but before I do, I just want to make sure I'm not headed down a dead-end street.
Thanks,
Steve
P.S. - The tests failed in both the 2.7.0b2-src and the what is currently in cvs.
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
I will take a look. Thanks! - C On Fri, 2003-10-17 at 17:04, Steve Jibson wrote:
Chris,
Sorry, I should have included more info, but my message was already a bit long.
I have run the tests on both Red Hat 7.3 and Red Hat 9 with the same result. I am using python 2.2.3. Here is the step-by-step for what I did (I'm going to err on the side of too much info here):
- check out latest Zope from cvs - cd Zope - ./configure - make - make instance - cd lib/python/Products/Sessions/stresstests - edit stresstestMultiThread.py - comment out line 38 - changed readiters and writeiters fro 100 to 10 on lines 117 and 118 - cp ~/Zope/lib/python/Products/Transience/tests/fauxtime.py . - python2 stresstestMultiThread.py
This time, on Red Hat 7.3, I actually got an exception. I'll include the output below.
Please let me know if there is ANYTHING I can do to help out with this one. I appreciate you looking into it.
Steve
------------------------------------------------------------------------
[stevej@elwood stresstests]$ python2 stresstestMultiThread.py testNonOverlappingBrowserIds (__main__.TestMultiThread) ... 4 values in toc waiting for 8 threads read conflict 4 values in toc read conflict read conflict read conflict waiting for 8 threads read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict 4 values in toc 4 values in toc 4 values in toc 4 values in toc 4 values in toc 4 values in toc 4 values in toc 4 values in toc read conflict read conflict read conflict read conflict read conflict read conflict waiting for 8 threads read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict waiting for 8 threads read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict 0 read conflict read conflict read conflict read conflict read conflict waiting for 7 threads read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict read conflict 2003-10-17T14:41:56 25 Exception in thread Thread-3: Traceback (most recent call last): File "//usr/lib/python2.2/threading.py", line 414, in __bootstrap self.run() File "stresstestMultiThread.py", line 164, in run self.run1() File "stresstestMultiThread.py", line 187, in run1 data = session_data_manager.getSessionData(create=1) File "/home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py", line 93, in getSessionData return self._getSessionDataObject(key) File "/home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py", line 180, in _getSessionDataObject ob = container.new_or_existing(key) File "/home/stevej/Zope/lib/python/Products/Transience/Transience.py", line 175, in new_or_existing item = self.get(key, notfound) File "/home/stevej/Zope/lib/python/Products/Transience/Transience.py", line 807, in get v = self._data[b].get(k, notfound) KeyError: 0
read conflict read conflict read conflict 2003-10-17T14:41:56 17 Exception in thread Thread-2: Traceback (most recent call last): File "//usr/lib/python2.2/threading.py", line 414, in __bootstrap self.run() File "stresstestMultiThread.py", line 164, in run self.run1() File "stresstestMultiThread.py", line 187, in run1 data = session_data_manager.getSessionData(create=1) File "/home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py", line 93, in getSessionData return self._getSessionDataObject(key) File "/home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py", line 180, in _getSessionDataObject ob = container.new_or_existing(key) File "/home/stevej/Zope/lib/python/Products/Transience/Transience.py", line 175, in new_or_existing item = self.get(key, notfound) File "/home/stevej/Zope/lib/python/Products/Transience/Transience.py", line 807, in get v = self._data[b].get(k, notfound) File "/home/stevej/Zope/lib/python/ZODB/Connection.py", line 561, in setstate p, serial = self._storage.load(oid, self._version) File "/home/stevej/Zope/lib/python/tempstorage/TemporaryStorage.py", line 94, in load s=self._index[oid] KeyError:
2003-10-17T14:41:56 18 Exception in thread Thread-4: Traceback (most recent call last): File "//usr/lib/python2.2/threading.py", line 414, in __bootstrap self.run() File "stresstestMultiThread.py", line 164, in run self.run1() File "stresstestMultiThread.py", line 187, in run1 data = session_data_manager.getSessionData(create=1) File "/home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py", line 93, in getSessionData return self._getSessionDataObject(key) File "/home/stevej/Zope/lib/python/Products/Sessions/SessionDataManager.py", line 180, in _getSessionDataObject ob = container.new_or_existing(key) File "/home/stevej/Zope/lib/python/Products/Transience/Transience.py", line 175, in new_or_existing item = self.get(key, notfound) File "/home/stevej/Zope/lib/python/Products/Transience/Transience.py", line 807, in get v = self._data[b].get(k, notfound) File "/home/stevej/Zope/lib/python/ZODB/Connection.py", line 561, in setstate p, serial = self._storage.load(oid, self._version) File "/home/stevej/Zope/lib/python/tempstorage/TemporaryStorage.py", line 94, in load s=self._index[oid] KeyError:
read conflict read conflict read conflict read conflict read conflict waiting for 4 threads read conflict read conflict read conflict read conflict read conflict read conflict
Chris McDonough wrote:
What OS? That test was written obviously to make sure the system works... others have complained about similar issues. I'd like to be able to track this down, but I can't make the test fail under Red Hat 7.3.
On Fri, 2003-10-17 at 15:56, Steve Jibson wrote:
We have been using Zope 2.7.0-beta2 and we have been having a bit of trouble with SESSION loosing data when we have more than one user on the system. These problems usually show up as KeyErrors when a particular data element in SESSION cannot be found. If I set the zserver threads to 1 in zope.conf, the problem seems to go away (but the system isn't really usable either). So I assumed it was a thread-safety problem in the SESSION code.
I decided to be brave and take a look at the code. To my surprize, I found a test file called stresstestMultiThread.py in the lib/python/Products/Sessions/stresstests directory that seemed to be the exact thing I wanted to test. It wouldn't run without a couple of minor modifications, but once I got it running, with more than three or four threads, the tests fail (never complete).
Is this test file still valid? It looks like it hasn't been modified for almost a year now. If it is valid, has anyone looked into getting the tests to pass. I will probably look into it more myself, but before I do, I just want to make sure I'm not headed down a dead-end street.
Thanks,
Steve
P.S. - The tests failed in both the 2.7.0b2-src and the what is currently in cvs.
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
participants (2)
-
Chris McDonough -
Steve Jibson