Conflict errors on BDBMinimal storage
Hi, whe are running portal unter zope2.7 with one zeo server and six frontend application servers. I have set up zeo to server two storages <filestorage main> path $INSTANCE_HOME/var/Data.fs </filestorage> <minimalstorage temp> envdir $INSTANCE_HOME/var/session </minimalstorage> temp should provide an shared session. But when is set up the frontends to use temp as /temp_folder, we see many ZODB conflict errors. The performance decreases and PosErrors appear quite often. Is there any why to get a shared session working. Bye Estartu ---------------------------------------------------------------------------- Gerhard Schmidt | Nick : estartu IRC : Estartu | PGP Public Key Fischbachweg 3 | Privat: estartu@augusta.de | auf Anfrage/ 86856 Hiltenfingen | Dienst: schmidt@ze.tu-muenchen.de | on request
On Thu, 2004-05-13 at 05:23, Gerhard Schmidt wrote:
Hi,
whe are running portal unter zope2.7 with one zeo server and six frontend application servers.
I have set up zeo to server two storages
<filestorage main> path $INSTANCE_HOME/var/Data.fs </filestorage>
<minimalstorage temp> envdir $INSTANCE_HOME/var/session </minimalstorage>
temp should provide an shared session. But when is set up the frontends to use temp as /temp_folder, we see many ZODB conflict errors. The performance decreases and PosErrors appear quite often.
BDBMininalStorage is no longer maintained and doesn't do any "conflict resolution". Even though you'll need to pack frequently, you might want to try a FileStorage instead. Regardless, using ZEO will presumably always generate more conflict errors than using a "local" ZODB because the commit time is longer, and thus there's a better chance that two threads will be committing at the same time. - C
On Mon, May 17, 2004 at 12:45:16PM -0400, Chris McDonough wrote:
On Thu, 2004-05-13 at 05:23, Gerhard Schmidt wrote:
Hi,
whe are running portal unter zope2.7 with one zeo server and six frontend application servers.
I have set up zeo to server two storages
<filestorage main> path $INSTANCE_HOME/var/Data.fs </filestorage>
<minimalstorage temp> envdir $INSTANCE_HOME/var/session </minimalstorage>
temp should provide an shared session. But when is set up the frontends to use temp as /temp_folder, we see many ZODB conflict errors. The performance decreases and PosErrors appear quite often.
BDBMininalStorage is no longer maintained and doesn't do any "conflict resolution".
Why this. I was told some time ago we should use BDBMinimalStorage to fix the conflict problems.
Even though you'll need to pack frequently, you might want to try a FileStorage instead.
There is no difference between BDB Storage and Filestorage regarding the conflicts.
Regardless, using ZEO will presumably always generate more conflict errors than using a "local" ZODB because the commit time is longer, and thus there's a better chance that two threads will be committing at the same time.
The bottom line is that there is no way of getting an shared session running. Why isn´t there an implementation of temporary storage for ZEO I dont care about conflit resolution. Its only session data. I simply need a shared session storage. But as Far as i can see the is no way of doing this. Are we the only site out there running Plone2 (Placeless transaltion Service) at that scale. How do others handle there Session Bye Estartu -------------------------------------------------------------------------- Gerhard Schmidt | Nick : estartu IRC : Estartu | Fischbachweg 3 | | PGP Public Key 86856 Hiltenfingen | estartu@augusta.de | auf Anfrage/ Tel: 08232 77 36 4 | | on request Fax: 08232 77 36 3 | |
On Mon, 2004-05-17 at 13:06, Gerhard Schmidt wrote:
BDBMininalStorage is no longer maintained and doesn't do any "conflict resolution".
Why this. I was told some time ago we should use BDBMinimalStorage to fix the conflict problems.
When was that and who told you so?
Even though you'll need to pack frequently, you might want to try a FileStorage instead.
There is no difference between BDB Storage and Filestorage regarding the conflicts.
OK.
Regardless, using ZEO will presumably always generate more conflict errors than using a "local" ZODB because the commit time is longer, and thus there's a better chance that two threads will be committing at the same time.
The bottom line is that there is no way of getting an shared session running.
It works doesn't it? You just get conflict errors, right? Conflict errors are normal at a certain level.
Why isn´t there an implementation of temporary storage for ZEO
There is, just use the "tempstorage" package from Zope.
I dont care about conflit resolution. Its only session data.
For better or worse, because it is based on ZODB, the implementation makes you need to care.
I simply need a shared session storage. But as Far as i can see the is no way of doing this.
I don't understand. You seem to have it working. Is performance the issue?
Are we the only site out there running Plone2 (Placeless transaltion Service) at that scale. How do others handle there Session
AFAIK Zope is the only appserver that even makes an attempt at allowing session data to be shared between servers like this. Most other appservers require you to set up "session affinity" in your load balancer and have a session database on each appserver. - C
On Mon, May 17, 2004 at 01:38:15PM -0400, Chris McDonough wrote:
On Mon, 2004-05-17 at 13:06, Gerhard Schmidt wrote:
BDBMininalStorage is no longer maintained and doesn't do any "conflict resolution".
Why this. I was told some time ago we should use BDBMinimalStorage to fix the conflict problems.
When was that and who told you so?
Even though you'll need to pack frequently, you might want to try a FileStorage instead.
There is no difference between BDB Storage and Filestorage regarding the conflicts.
OK.
Regardless, using ZEO will presumably always generate more conflict errors than using a "local" ZODB because the commit time is longer, and thus there's a better chance that two threads will be committing at the same time.
The bottom line is that there is no way of getting an shared session running.
It works doesn't it? You just get conflict errors, right? Conflict errors are normal at a certain level.
No It doesn`t. At the Moment we run with single sessions on each frontend we have rewitten most ob the code using sessions to store ther information in cockies. But that´s not realy a good solution.
Why isn´t there an implementation of temporary storage for ZEO
There is, just use the "tempstorage" package from Zope.
I didn´t find a way to use Tempstorage in zeo. <temporarystorage> name temporary storage for sessioning </temporarystorage> didn´t work. How do i Set up a temporarystorage in zeo.
I dont care about conflit resolution. Its only session data.
For better or worse, because it is based on ZODB, the implementation makes you need to care.
Than maybe the sessioncode should be redesigned not to use ZODB as storage but store ther information in some different system maybe a RDMB or some other system able to handle the kind of usage.
I simply need a shared session storage. But as Far as i can see the is no way of doing this.
I don't understand. You seem to have it working. Is performance the issue?
The Performance decrase is there but the Problem is far deeper. If an read conflict isn´t resolved after three retries an Exception is raised an the request is aborted. And thats the real problem.
Are we the only site out there running Plone2 (Placeless transaltion Service) at that scale. How do others handle there Session
AFAIK Zope is the only appserver that even makes an attempt at allowing session data to be shared between servers like this. Most other appservers require you to set up "session affinity" in your load balancer and have a session database on each appserver.
and nealy every system failed to do so. How do you implement this session affinity. Based on source IP. At the moment every large ISP uses multiple Proxyserver so every request comes from a different IP so the affinity doesn´t work for them. The Proxyserver provided by our Computing Center is also split on at least 8 Hosts, so everyone using this proxy would lose session infomation, getting very strange effekts. Bye Estartu -------------------------------------------------------------------------- Gerhard Schmidt | Nick : estartu IRC : Estartu | Fischbachweg 3 | | PGP Public Key 86856 Hiltenfingen | estartu@augusta.de | auf Anfrage/ Tel: 08232 77 36 4 | | on request Fax: 08232 77 36 3 | |
On Mon, 2004-05-17 at 14:57, Gerhard Schmidt wrote:
Why isn´t there an implementation of temporary storage for ZEO
There is, just use the "tempstorage" package from Zope.
I didn´t find a way to use Tempstorage in zeo.
<temporarystorage> name temporary storage for sessioning </temporarystorage>
didn´t work. How do i Set up a temporarystorage in zeo.
Good question. ;-) It involves changing the ZEO configuration machinery to recognize the tempstorage package. The steps to do this I couldn't tell you without actually doing it, however.
I dont care about conflit resolution. Its only session data.
For better or worse, because it is based on ZODB, the implementation makes you need to care.
Than maybe the sessioncode should be redesigned not to use ZODB as storage but store ther information in some different system maybe a RDMB or some other system able to handle the kind of usage.
Zope doesn't ship with a useful and maintained relational database. You can implement your own SQL-based TransientObjectContainer using the interfaces defined in Transience/Interfaces.py if you wish, or use Anthony Baxter's SQLSession product. It would be nice if there was an alternate implementation that used a relational database like this.
I simply need a shared session storage. But as Far as i can see the is no way of doing this.
Of course there is, it's just not built in. ;-)
I don't understand. You seem to have it working. Is performance the issue?
The Performance decrase is there but the Problem is far deeper. If an read conflict isn´t resolved after three retries an Exception is raised an the request is aborted. And thats the real problem.
I see. I'm not sure switching to TemporaryStorage will help much. I would first try using a FileStorage and make sure to put the Zope "lib/python" directory on the ZEO server's Python's PYTHONPATH (to allow for conflict resolution under ZEO, ZEO must be able to import the Products package and subpackages therein). I don't know if this will help very much, but it's worth a shot.
Are we the only site out there running Plone2 (Placeless transaltion Service) at that scale. How do others handle there Session
AFAIK Zope is the only appserver that even makes an attempt at allowing session data to be shared between servers like this. Most other appservers require you to set up "session affinity" in your load balancer and have a session database on each appserver.
and nealy every system failed to do so. How do you implement this session affinity. Based on source IP. At the moment every large ISP uses multiple Proxyserver so every request comes from a different IP so the affinity doesn´t work for them. The Proxyserver provided by our Computing Center is also split on at least 8 Hosts, so everyone using this proxy would lose session infomation, getting very strange effekts.
Yes. It's a hard problem. - C
On 18/05/2004, at 6:57 AM, Gerhard Schmidt wrote:
On Mon, May 17, 2004 at 01:38:15PM -0400, Chris McDonough wrote:
AFAIK Zope is the only appserver that even makes an attempt at allowing session data to be shared between servers like this. Most other appservers require you to set up "session affinity" in your load balancer and have a session database on each appserver.
and nealy every system failed to do so. How do you implement this session affinity. Based on source IP. At the moment every large ISP uses multiple Proxyserver so every request comes from a different IP so the affinity doesn´t work for them. The Proxyserver provided by our Computing Center is also split on at least 8 Hosts, so everyone using this proxy would lose session infomation, getting very strange effekts.
Perhaps you can get some relief by using pound as your load balancer. It is quite capable at sniffing out user sessions including watching the Cookie header. See http://www.apsis.ch/pound/ Michael
Hi Chris, hi folks. thanks for the hint with minimal storage. however the same problem occurs with filestorage and is even worse. we know very well, that we cannot avoid the appearance of conflict errors in particular with zeo, and we understand what causes them. what wonders us is, that we find some contributions about this errors in the mailing lists, but no one seems to have a real problem with it. but we have! we are not worried about some errors appearing in a log, but we are using plone2 witch currently 30 concurrent authenticated users, so that this error occurs permanently, thus decreasing system performance due to several retries and moreover leading to very frequent nasty 'POS.Exception Errors' in the browser, whenever four successive tries fail (almost every 100. request). this is not acceptable, and i wonder wheter there are any plone sites with more than a handful of authenticated users that experience the same problem. we found that one (there are more) product responsible for it is the placeless translation service, which heavily writes to the session. so we desperately need a solution for this problem, maybe uncouple the session of the locking mechanism or whatever, and this should be a problem that should be addressed by the core team. because this is a strong reason (and so far the only one) to avoid using zope/plone in a large user environment. i hope some of the community sees the importance of this. Regards Thomas
On Thu, 2004-05-13 at 05:23, Gerhard Schmidt wrote:
Hi,
whe are running portal unter zope2.7 with one zeo server and six frontend application servers.
I have set up zeo to server two storages
<filestorage main> path $INSTANCE_HOME/var/Data.fs </filestorage>
<minimalstorage temp> envdir $INSTANCE_HOME/var/session </minimalstorage>
temp should provide an shared session. But when is set up the frontends to use temp as /temp_folder, we see many ZODB conflict errors. The performance decreases and PosErrors appear quite often.
BDBMininalStorage is no longer maintained and doesn't do any "conflict resolution". Even though you'll need to pack frequently, you might want to try a FileStorage instead.
Regardless, using ZEO will presumably always generate more conflict errors than using a "local" ZODB because the commit time is longer, and thus there's a better chance that two threads will be committing at the same time.
- C
Hi Dr. Wagner,
thanks for the hint with minimal storage. however the same problem occurs with filestorage and is even worse.
we know very well, that we cannot avoid the appearance of conflict errors in particular with zeo, and we understand what causes them.
what wonders us is, that we find some contributions about this errors in the mailing lists, but no one seems to have a real problem with it. but we have! we are not worried about some errors appearing in a log, but we are using plone2 witch currently 30 concurrent authenticated users, so that this error occurs permanently, thus decreasing system performance due to several retries and moreover leading to very frequent nasty 'POS.Exception Errors' in the browser, whenever four successive tries fail (almost every 100. request).
this is not acceptable, and i wonder wheter there are any plone sites with more than a handful of authenticated users that experience the same problem. we found that one (there are more) product responsible for it is the placeless translation service, which heavily writes to the session.
Are you using ZEO for your session storage or are you using a local storage? (You hint at it above but you don't actually say one way or the other.)
so we desperately need a solution for this problem, maybe uncouple the session of the locking mechanism or whatever, and this should be a problem that should be addressed by the core team. because this is a strong reason (and so far the only one) to avoid using zope/plone in a large user environment.
I also need reliable sessioning for my own customers, so I am working towards fixing the issues I know about. But the issues I know about may not be the same issues as those that you want fixed. You can ensure your issues are addressed by devoting resources towards the problem, be that time+expertise or money or both. - C
Chris McDonough wrote:
Hi Dr. Wagner,
thanks for the hint with minimal storage. however the same problem occurs with filestorage and is even worse.
we know very well, that we cannot avoid the appearance of conflict errors in particular with zeo, and we understand what causes them.
what wonders us is, that we find some contributions about this errors in the mailing lists, but no one seems to have a real problem with it. but we have! we are not worried about some errors appearing in a log, but we are using plone2 witch currently 30 concurrent authenticated users, so that this error occurs permanently, thus decreasing system performance due to several retries and moreover leading to very frequent nasty 'POS.Exception Errors' in the browser, whenever four successive tries fail (almost every 100. request).
this is not acceptable, and i wonder wheter there are any plone sites with more than a handful of authenticated users that experience the same problem. we found that one (there are more) product responsible for it is the placeless translation service, which heavily writes to the session.
Are you using ZEO for your session storage or are you using a local storage? (You hint at it above but you don't actually say one way or the other.)
so we desperately need a solution for this problem, maybe uncouple the session of the locking mechanism or whatever, and this should be a problem that should be addressed by the core team. because this is a strong reason (and so far the only one) to avoid using zope/plone in a large user environment.
I also need reliable sessioning for my own customers, so I am working towards fixing the issues I know about. But the issues I know about may not be the same issues as those that you want fixed. You can ensure your issues are addressed by devoting resources towards the problem, be that time+expertise or money or both.
I recently reworked Anthony Baxter's SQLSessions product for a consulting customer, essentially modernizing the code and making it compliant with the new Sessions interface (I dropped the bits which do the work currently done by BrowserIDManager). It took me about six hours or so, including writing unit tests. Unfortunately, I don't have access to that code, due to the nature of the consulting arrangement. What I would like to do is revisit the effort, using the new PluginRegistry code released as part of the PluggableAuthService release; I am, however, pretty well booked up for the next few weeks. I would be glad to advise anybody who felt called to take on such an effort, however. Tres. -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com
On Tue, 2004-05-18 at 14:15, Tres Seaver wrote:
I recently reworked Anthony Baxter's SQLSessions product for a consulting customer, essentially modernizing the code and making it compliant with the new Sessions interface (I dropped the bits which do the work currently done by BrowserIDManager). It took me about six hours or so, including writing unit tests. Unfortunately, I don't have access to that code, due to the nature of the consulting arrangement.
What I would like to do is revisit the effort, using the new PluginRegistry code released as part of the PluggableAuthService release; I am, however, pretty well booked up for the next few weeks. I would be glad to advise anybody who felt called to take on such an effort, however.
I would also like to see this happen. I don't have the time to help implement this right now, but I may be able to work it in to an ongoing engagement at some point in the next few months. - C
Chris McDonough wrote:
On Tue, 2004-05-18 at 14:15, Tres Seaver wrote:
I recently reworked Anthony Baxter's SQLSessions product for a consulting customer, essentially modernizing the code and making it compliant with the new Sessions interface (I dropped the bits which do the work currently done by BrowserIDManager). It took me about six hours or so, including writing unit tests. Unfortunately, I don't have access to that code, due to the nature of the consulting arrangement.
What I would like to do is revisit the effort, using the new PluginRegistry code released as part of the PluggableAuthService release; I am, however, pretty well booked up for the next few weeks. I would be glad to advise anybody who felt called to take on such an effort, however.
I would also like to see this happen. I don't have the time to help implement this right now, but I may be able to work it in to an ongoing engagement at some point in the next few months.
- C
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
I do not know whether I understand this email correctly. So mabe I am off topic. We would like to use thre PluggableAuthService to replace a userfolder that authenticates against a an email server (calling a web service to do so) and then caches authentification and other user info in a session. So we would like to dig into what both of you seem not to have time to do. How can we get at PluggableAuthService? I did not find it anywhere. And we would of course gladly accept Tres offer for advise. Robert
robert rottermann wrote:
On Tue, 2004-05-18 at 14:15, Tres Seaver wrote:
I recently reworked Anthony Baxter's SQLSessions product for a consulting customer, essentially modernizing the code and making it compliant with the new Sessions interface (I dropped the bits which do the work currently done by BrowserIDManager). It took me about six hours or so, including writing unit tests. Unfortunately, I don't have access to that code, due to the nature of the consulting arrangement.
What I would like to do is revisit the effort, using the new PluginRegistry code released as part of the PluggableAuthService release; I am, however, pretty well booked up for the next few weeks. I would be glad to advise anybody who felt called to take on such an effort, however.
I do not know whether I understand this email correctly. So mabe I am off topic. We would like to use thre PluggableAuthService to replace a userfolder that authenticates against a an email server (calling a web service to do so) and then caches authentification and other user info in a session. So we would like to dig into what both of you seem not to have time to do. How can we get at PluggableAuthService? I did not find it anywhere.
The PluggableAuthService product is available in CVS: http://cvs.zope.org/Products/PluggableAuthService It depends on the PluginRegistry product: http://cvs.zope.org/Products/PluginRegistry Both are "releasable", but not yet released (I just reminded Zac to cut the release tarballs). PAS has a SessionAuthHelper plugin which should serve as a model for the part of your problem which revolves around the session; you would need to write an "authenticator" plugin which validated credentials against your web service.
And we would of course gladly accept Tres offer for advise.
I was referring to using the PluginRegistry, which is a lower-level utility, to (re)write the SQLSessions product, which needs to be "pluggable" in order to accomodate different backends. The PIR is a Zope 2 analog to the Zope 3 "local utility" service: it stores a perstent map of the plugins registered / activated for a set of application-defined plugin interfaces. Tres. -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com
Tres Seaver wrote:
I recently reworked Anthony Baxter's SQLSessions product for a consulting customer
<...snip...>
What I would like to do is revisit the effort, using the new PluginRegistry code released as part of the PluggableAuthService release; I am, however, pretty well booked up for the next few weeks. I would be glad to advise anybody who felt called to take on such an effort, however.
<...snip...> I actually need this whithin the next couple of weeks - only one caveat: I'd have to do a serious read-up on sessioning, which I don't understand at all and the PluggableAuthService product, which I never heard of until I read about it. We have something similar to SQL-Sessions. However, our security machinery is built on top of Zopes own (in parallell not replacing it). It is loosely modelled around a mix of Zope's way of doing things with a hint of XUF (we couldn't use XUF at the time, because it wasn't stable enough back then). One of the goals wiht the current rewrite of one of our apps is to go back doing things the Zope-way, by using Zope's own security mechanisms. We have working solutions for authentication, but the sessioning part remains - especically since we, like in Steve's setup expire sessions on the server rather than on the browser end. So... any hints on where to begin reading for the completely-lost newbie? /dario -- -- ------------------------------------------------------------------- Dario Lopez-Kästen, IT Systems & Services Chalmers University of Tech.
Dario Lopez-Kästen wrote:
Tres Seaver wrote:
I recently reworked Anthony Baxter's SQLSessions product for a consulting customer
<...snip...>
What I would like to do is revisit the effort, using the new PluginRegistry code released as part of the PluggableAuthService release; I am, however, pretty well booked up for the next few weeks. I would be glad to advise anybody who felt called to take on such an effort, however.
<...snip...>
I actually need this whithin the next couple of weeks - only one caveat: I'd have to do a serious read-up on sessioning, which I don't understand at all and the PluggableAuthService product, which I never heard of until I read about it.
It wouldn't necessarily require PAS, merely the PluginRegistry which was released with it. The reason for this requirement is that a "SQLSessionNG" product needs to have a set of plugins, which will typically be customized on a per-site basis to fit whatever backend RDBMS schema is in use. Perhaps we should call the product "PluggableSessions", as there isn't any particular requirement that the backend be SQL based.
We have something similar to SQL-Sessions. However, our security machinery is built on top of Zopes own (in parallell not replacing it). It is loosely modelled around a mix of Zope's way of doing things with a hint of XUF (we couldn't use XUF at the time, because it wasn't stable enough back then).
One of the goals wiht the current rewrite of one of our apps is to go back doing things the Zope-way, by using Zope's own security mechanisms. We have working solutions for authentication, but the sessioning part remains - especically since we, like in Steve's setup expire sessions on the server rather than on the browser end.
PAS does have a SessionAuthHelper which stashes credentials in the session; you could probably wrap up your current authentication solution as an IAuthenticationPlugin plugin for it.
So... any hints on where to begin reading for the completely-lost newbie?
Tres. -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com
On Tue, 2004-05-18 at 14:58, Chris McDonough wrote:
On Tue, 2004-05-18 at 14:15, Tres Seaver wrote:
I recently reworked Anthony Baxter's SQLSessions product for a consulting customer, essentially modernizing the code and making it compliant with the new Sessions interface (I dropped the bits which do the work currently done by BrowserIDManager). It took me about six hours or so, including writing unit tests. Unfortunately, I don't have access to that code, due to the nature of the consulting arrangement.
What I would like to do is revisit the effort, using the new PluginRegistry code released as part of the PluggableAuthService release; I am, however, pretty well booked up for the next few weeks. I would be glad to advise anybody who felt called to take on such an effort, however.
I would also like to see this happen. I don't have the time to help implement this right now, but I may be able to work it in to an ongoing engagement at some point in the next few months.
In my testing, I'm able to see that naive sessioning applications (ones that hit the session on every request, whether they really need to or not) seem to fail down due to the "abort after 3 conflicts in a row" hardcoded policy under extremely high load (~ 40-60 req/sec or so). This will happen regardless of what code I put in the sessioning machinery; attempting to prevent it there is a lost cause: it gets a bit like trying to prove the Heisenberg Uncertainty Principal or trying to predict the future. As a result, I am coming to believe that along with the "errors as part of main" transaction patch for 2.7.1, I should also make the retry-on-conflict-error policy pluggable for those who really desperately need to slow their Zope systems to a conflict-induced crawl. They can bump it up to 100 as far as I care and if it works for them, great. The more selective people can wait around for a different (non-ZODB-based) sessioning implementation or change their code to not pound the snot out of the sessioning machinery unnecessarily. As far as I'm concerned, the new Transience stuff is looking really good for the common case. I haven't been able to reproduce any of the corruption issues that happened with the older implementation. - C
On 19/05/2004, at 8:36 AM, Chris McDonough wrote:
As a result, I am coming to believe that along with the "errors as part of main" transaction patch for 2.7.1, I should also make the retry-on-conflict-error policy pluggable for those who really desperately need to slow their Zope systems to a conflict-induced crawl. They can bump it up to 100 as far as I care and if it works for them, great.
Or this could be used to avoid retries altogether. And at the very least it would advertise the fact that zope does retry requests. Michael.
On Tue, 2004-05-18 at 16:36, Chris McDonough wrote:
In my testing, I'm able to see that naive sessioning applications (ones that hit the session on every request, whether they really need to or not) seem to fail down due to the "abort after 3 conflicts in a row" hardcoded policy under extremely high load (~ 40-60 req/sec or so). This will happen regardless of what code I put in the sessioning machinery; attempting to prevent it there is a lost cause: it gets a bit like trying to prove the Heisenberg Uncertainty Principal or trying to predict the future.
Just as a followup: when I observed this, it was because I had all of my extra debugging code turned on. True to the Uncertainty Principle, the execution of the debugging code caused a raft of extra conflicts. Turning off the debugging code leads to a much rosier scenario. In cases where it was raising conflicts to the end user, it now operates without doing so. I still need to do some really hardcore testing of the thing to see exactly where it buckles, but I can no longer provoke it trivially.
As a result, I am coming to believe that along with the "errors as part of main" transaction patch for 2.7.1, I should also make the retry-on-conflict-error policy pluggable for those who really desperately need to slow their Zope systems to a conflict-induced crawl. They can bump it up to 100 as far as I care and if it works for them, great. The more selective people can wait around for a different (non-ZODB-based) sessioning implementation or change their code to not pound the snot out of the sessioning machinery unnecessarily.
Michael Dunstan also had another good idea. At least one knob exists in the source code to increase/decrease the "resolution" of a part of the sessioning/transience code. Basically this knob allows you to choose a tradeoff between the frequency of a class of writes done by the session code and the "accuracy" of the session "finalization". Setting the knob higher will decrease the aggregate number of database writes that would need to be done over any given period of time in order to keep sessions "current", which might lead to fewer conflicts. On the flip side, it would mean that the session might not be "finalized" on time (sessions are "finalized" when they are garbage collected; sessions are only allowed to be garbage collected every "n" seconds where n is the knob's setting). This would typically be meaningful only to people who use the "Script to call at session deletion time" feature of sessions; that script might be called much later than you think it might be called if you set this value high enough. There's currently no guarantee that it will be called exactly when a session expires even with the default value, but setting the resolution higher will decrease that probability even further. Since that feature isn't used very much (at least I don't think it is), I think this is an ok tradeoff for a lot of people. I'd need to make sure that it actually helps something before doing so, but assuming it did, it would be put into the config file for 2.7.1 as something like "session-resolution-seconds" or somesuch. For those of you with conflict problems now, this knob is the PERIOD variable in Transience.py... you might be able to get away with setting it higher now to see if that helps anything, especially those people trying to use sessions+ZEO. It's preset to 20, which means 20 seconds. Try 120 instead, or something around there. Just don't set it to a number higher than the timeout value for the TOC (which is in minutes, remember to multiply it by 60) or all sorts of insanity will happen. You'll need to restart Zope and recreate your transient object container after doing this, as doing this will effectively invalidate all data previously in the container and using old data left over from running it under the default PERIOD value might make it go a little insane too.
As far as I'm concerned, the new Transience stuff is looking really good for the common case. I haven't been able to reproduce any of the corruption issues that happened with the older implementation.
This is still true (it's been almost an hour since I posted it, I think that's a new record!) - c
Gerhard Schmidt wrote at 2004-5-13 11:23 +0200:
... Is there any why to get a shared session working.
You know that the ZODB is not designed for high frequency concurrent writes? When you want to reduce conflicts you will need to try hard to minimize concurrent writes. I have a (non-open) "Transience" implementation that writes much less then Zope's standard implementation. However, due to a ZEO weakness, its conflict resolution currently does not work (ZEO no longer supports "INSTANCE_HOME"). -- Dieter
On Mon, May 17, 2004 at 07:48:22PM +0200, Dieter Maurer wrote:
Gerhard Schmidt wrote at 2004-5-13 11:23 +0200:
... Is there any why to get a shared session working.
You know that the ZODB is not designed for high frequency concurrent writes?
When you want to reduce conflicts you will need to try hard to minimize concurrent writes.
Thats not that easy. Because we use many Thrid Party Products many of them use session quite often (namely Placeless Translation Service).
I have a (non-open) "Transience" implementation that writes much less then Zope's standard implementation. However, due to a ZEO weakness, its conflict resolution currently does not work (ZEO no longer supports "INSTANCE_HOME").
Is that a bug or intetional. Bye Estartu -------------------------------------------------------------------------- Gerhard Schmidt | Nick : estartu IRC : Estartu | Fischbachweg 3 | | PGP Public Key 86856 Hiltenfingen | estartu@augusta.de | auf Anfrage/ Tel: 08232 77 36 4 | | on request Fax: 08232 77 36 3 | |
Gerhard Schmidt wrote at 2004-5-17 21:01 +0200:
...
However, due to a ZEO weakness, its conflict resolution currently does not work (ZEO no longer supports "INSTANCE_HOME").
Is that a bug or intetional.
I expect it to be a bug. I plan to fix it soon -- but currently, I have lots of things to do. May take some months... -- Dieter
participants (8)
-
Chris McDonough -
Dario Lopez-Kästen -
Dieter Maurer -
Dr. Thomas Wagner -
Gerhard Schmidt -
Michael Dunstan -
robert rottermann -
Tres Seaver