thread safety in external methods
How do I do thread safety inside an External method. I want to access a global singleton as def __ZGA(): if not hasattr(sys.modules['__main__'],'__ZGA'): #unsafe here... Z=ZGA_calculator() # initialise things to do with Z sys.modules['__main__'].__dict__['__ZGA']=Z return sys.modules['__main__'].__dict__['__ZGA'] the problem is controlling access to sys.modules['__main__'].__dict__['__ZGA']. I can't have a global lock inside my external method as I get a different environment with each call (which is why I need the singleton in the first place). It would be sufficient for my purposes if I could guarantee to get certain code run just once at startup time. Do I have to make a product or is there some easier way to do this. -- Robin Becker
Robin Becker wrote:
How do I do thread safety inside an External method.
I want to access a global singleton as
def __ZGA(): if not hasattr(sys.modules['__main__'],'__ZGA'): #unsafe here... Z=ZGA_calculator() # initialise things to do with Z sys.modules['__main__'].__dict__['__ZGA']=Z return sys.modules['__main__'].__dict__['__ZGA']
the problem is controlling access to sys.modules['__main__'].__dict__['__ZGA']. I can't have a global lock inside my external method as I get a different environment with each call (which is why I need the singleton in the first place).
It would be sufficient for my purposes if I could guarantee to get certain code run just once at startup time.
When Zope starts up it imports all the modules containing external methods. It might be possible to place code directly in the module for execution upon import. I'm not 100% certain, however, but I don't see why this standard python feature would have been disabled. -Michel
Do I have to make a product or is there some easier way to do this. -- Robin Becker
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://www.zope.org/mailman/listinfo/zope-dev
(To receive general Zope announcements, see: http://www.zope.org/mailman/listinfo/zope-announce
For non-developer, user-level issues, zope@zope.org, http://www.zope.org/mailman/listinfo/zope )
In article <37BEAC6C.3DDDA7EC@digicool.com>, Michel Pelletier <michel@digicool.com> writes
Robin Becker wrote:
...
When Zope starts up it imports all the modules containing external methods. It might be possible to place code directly in the module for execution upon import.
I'm not 100% certain, however, but I don't see why this standard python feature would have been disabled.
-Michel ... I tried that before and found I was getting a different version of my module globals somehow when the function was called. I already had a job queue set up, but it was being fooled somehow.
My example is as follows; the following is in root/Extensions (parallel to root/var) #### external method doesn't have proper globals import time JQ=0 class Thing: def __init__(self): global JQ self.I=JQ JQ=JQ+1 def val(self): global JQ import time time.sleep(10) return "%s I=%d JQ=%d"%(str(self),self.I,JQ) thing=Thing() def drongo(self): return thing.val() I add this as function test/drongo with function drongo when I run this twice (nearly simultaneously) I get <?.Thing instance at 1ee9aa0> I=0 JQ=1 from one and <?.Thing instance at 1eb6e30> I=0 JQ=1 from the other a third run gives <?.Thing instance at 1eb6e30> I=0 JQ=1 so I had two versions of the global at one point.
Do I have to make a product or is there some easier way to do this. -- Robin Becker
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://www.zope.org/mailman/listinfo/zope-dev
(To receive general Zope announcements, see: http://www.zope.org/mailman/listinfo/zope-announce
For non-developer, user-level issues, zope@zope.org, http://www.zope.org/mailman/listinfo/zope )
-- Robin Becker
Michel Pelletier wrote:
Robin Becker wrote:
How do I do thread safety inside an External method.
I want to access a global singleton as
def __ZGA(): if not hasattr(sys.modules['__main__'],'__ZGA'): #unsafe here... Z=ZGA_calculator() # initialise things to do with Z sys.modules['__main__'].__dict__['__ZGA']=Z return sys.modules['__main__'].__dict__['__ZGA']
the problem is controlling access to sys.modules['__main__'].__dict__['__ZGA']. I can't have a global lock inside my external method as I get a different environment with each call (which is why I need the singleton in the first place).
It would be sufficient for my purposes if I could guarantee to get certain code run just once at startup time.
When Zope starts up it imports all the modules containing external methods.
No it doesn't. It imports products at startup. It *loads* external methods when it needs to. Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
Robin Becker wrote:
How do I do thread safety inside an External method.
I want to access a global singleton as
def __ZGA(): if not hasattr(sys.modules['__main__'],'__ZGA'): #unsafe here... Z=ZGA_calculator() # initialise things to do with Z sys.modules['__main__'].__dict__['__ZGA']=Z return sys.modules['__main__'].__dict__['__ZGA']
the problem is controlling access to sys.modules['__main__'].__dict__['__ZGA']. I can't have a global lock inside my external method as I get a different environment with each call (which is why I need the singleton in the first place).
It would be sufficient for my purposes if I could guarantee to get certain code run just once at startup time.
Is this actually necessary? Is it cirtical that only one ZGA_calculator gets created? If not, you don't really have a thread-safety problem here. If it is, then you could protect the section above with a lock: import thread zgalock=tread.allocate_lock() def __ZGA(): zgaloc.acquire() try: if not hasattr(sys.modules['__main__'],'__ZGA'): #unsafe here... Z=ZGA_calculator() # initialise things to do with Z sys.modules['__main__'].__dict__['__ZGA']=Z return sys.modules['__main__'].__dict__['__ZGA'] finally: zgaloc.release()
Do I have to make a product or is there some easier way to do this.
I think that a product would be easier. With a product, you could simply put the above code in the product __init__ module. BTW, why do you want to hack __main__? Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
In article <37CE7664.395D31F5@digicool.com>, Jim Fulton <jim@digicool.com> writes
Robin Becker wrote:
How do I do thread safety inside an External method.
...
e).
It would be sufficient for my purposes if I could guarantee to get certain
code run just once
at startup time.
Is this actually necessary? Is it cirtical that only one ZGA_calculator gets created? If not, you don't really have a thread-safety problem here. If it is, then you could protect the section above with a lock:
import thread zgalock=tread.allocate_lock() def __ZGA(): zgaloc.acquire() try: if not hasattr(sys.modules['__main__'],'__ZGA'): #unsafe here... Z=ZGA_calculator() # initialise things to do with Z sys.modules['__main__'].__dict__['__ZGA']=Z return sys.modules['__main__'].__dict__['__ZGA'] finally: zgaloc.release()
Do I have to make a product or is there some easier way to do this.
I think that a product would be easier. With a product, you could simply put the above code in the product __init__ module.
BTW, why do you want to hack __main__?
Jim
I do indeed need only a single calculator. My process is so big that I can afford only one running at a time. I therefore implement queuing etc via the singleton. My original reply to Michel shows that doing as you suggest in the external method isn't sufficient. If two calls hit the method together then two locks may be created simultaneously and then the lock doesn't work. It seems that each invocation of an external method gets a private global space. I have to move the lock into an imported file to get a lock to work. And in fact you're right as soon as I have the imported file I can keep __ZGA there and not in __main__. -- Robin Becker
Robin Becker wrote:
My original reply to Michel shows that doing as you suggest in the external method isn't sufficient. If two calls hit the method together then two locks may be created simultaneously and then the lock doesn't work.
Oops. Good point.
It seems that each invocation of an external method gets a private global space.
Right. External method files are *not* modules. They should not depend on usual module semantics.
I have to move the lock into an imported file to get a lock to work. And in fact you're right as soon as I have the imported file I can keep __ZGA there and not in __main__.
The right answer in a case like this is to use a product, not an external method. Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
In article <37CFC7B4.FFF5AA86@digicool.com>, Jim Fulton <jim@digicool.com> writes
Robin Becker wrote:
...
The right answer in a case like this is to use a product, not an external method.
Jim
Well I didn't know anything about products and in fact it seemed a bit like overkill when I read about them. I believed at first that external methods were like bits of real python called via XML or dtml in fact there's quite a lot in them that's different. Can a product just export methods? I guess it can. -- Robin Becker
participants (3)
-
Jim Fulton -
Michel Pelletier -
Robin Becker