I've got a product that has a little pool of database connections. These are low level DCOracle2 objects, not Zope DB connections. These live in a hash table-like cache living in a volatile attribute of a persistent product. Something like this: myproduct _v_mycache conn1 conn2 conn3 The cache periodically reaps old connections. The product insures the cache exists before it uses it, as it is volatile and might not exist at any point in time. Here's my problem: Occasionally, my _v_mycahce is empty when I didn't clear it. It looks like my product object gets dropped from memory, then reloaded, resulting in a missing cache. If this is true then what I need is an class or attribute that is persistent for the server lifetime. It looks like all I can get from Zope is permanent persistent objects or volatile objects. So my questions are: 1) Can I put a trap somewhere on my object so I know when it gets purged or loaded? Just a simple "sys.stderr.write('loading\n')" would be enough to confirm my suspicions. 2) Is there a server-lifetime class of objects? Is there a workaround to get there? Can I force my object and its attributes to live in memory and not get purged until shutdown? Thanks in advance. I think I can work around this with traditional coding kung-fu, but it'll be a pain. A solution like 'set this attribute to true' or 'subclass from twiddle' would make me really happy. Charlie.
Occasionally, my _v_mycahce is empty when I didn't clear it. It looks like my product object gets dropped from memory, then reloaded, resulting in a missing cache.
Or you could be an a different thread since _v_ attribute is actually thread specific.
2) Is there a server-lifetime class of objects? Is there a workaround to get there? Can I force my object and its attributes to live in memory and not get purged until shutdown?
I havent tried it yet, but in 2.5.1 you could use the Temporary Folder.... -- Andy McKay Agmweb Consulting http://www.agmweb.ca ----- Original Message ----- From: "Charlie Reiman" <creiman@kefta.com> To: <zope@zope.org> Sent: Thursday, August 15, 2002 5:55 PM Subject: [Zope] Persistence Problem
I've got a product that has a little pool of database connections. These are low level DCOracle2 objects, not Zope DB connections. These live in a hash table-like cache living in a volatile attribute of a persistent product.
Something like this:
myproduct _v_mycache conn1 conn2 conn3
The cache periodically reaps old connections. The product insures the cache exists before it uses it, as it is volatile and might not exist at any point in time. Here's my problem:
Occasionally, my _v_mycahce is empty when I didn't clear it. It looks like my product object gets dropped from memory, then reloaded, resulting in a missing cache. If this is true then what I need is an class or attribute that is persistent for the server lifetime. It looks like all I can get from Zope is permanent persistent objects or volatile objects.
So my questions are:
1) Can I put a trap somewhere on my object so I know when it gets purged or loaded? Just a simple "sys.stderr.write('loading\n')" would be enough to confirm my suspicions.
2) Is there a server-lifetime class of objects? Is there a workaround to get there? Can I force my object and its attributes to live in memory and not get purged until shutdown?
Thanks in advance. I think I can work around this with traditional coding kung-fu, but it'll be a pain. A solution like 'set this attribute to true' or 'subclass from twiddle' would make me really happy.
Charlie.
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
as andy mentioned, _v_ volatile attributes are specific to a thread (well, technically there specific to the underlying zodb connection). they remain only as long as the object remains in memory/cache. there is no notice when object is evicted from cache/memory. the object returns to a ghost state, which has no instance state till its accessed again. if you really want global variables that exist for the lifetime of the server, use module level attributes, they are shared by all threads, you will need to do your own locking. -kapil On Thursday 15 August 2002 05:55 pm, Charlie Reiman wrote:
I've got a product that has a little pool of database connections. These are low level DCOracle2 objects, not Zope DB connections. These live in a hash table-like cache living in a volatile attribute of a persistent product.
Something like this:
myproduct _v_mycache conn1 conn2 conn3
The cache periodically reaps old connections. The product insures the cache exists before it uses it, as it is volatile and might not exist at any point in time. Here's my problem:
Occasionally, my _v_mycahce is empty when I didn't clear it. It looks like my product object gets dropped from memory, then reloaded, resulting in a missing cache. If this is true then what I need is an class or attribute that is persistent for the server lifetime. It looks like all I can get from Zope is permanent persistent objects or volatile objects.
So my questions are:
1) Can I put a trap somewhere on my object so I know when it gets purged or loaded? Just a simple "sys.stderr.write('loading\n')" would be enough to confirm my suspicions.
2) Is there a server-lifetime class of objects? Is there a workaround to get there? Can I force my object and its attributes to live in memory and not get purged until shutdown?
Thanks in advance. I think I can work around this with traditional coding kung-fu, but it'll be a pain. A solution like 'set this attribute to true' or 'subclass from twiddle' would make me really happy.
Charlie.
Thanks everyone. This dawned on me as I was on the train riding home. I also found __setstate__ and __getstate__ if I want to mess with the loading hooks (I don't). http://www.zopelabs.com/cookbook/1002251996 The Zope Developer's guide does talk about module variables. But all it says is: "Objects stored in modules but not in the ZODB are not persistent and not-thread safe. In general it's not a good idea to store data (as opposed to functions, and class definitions) in modules when using ZODB." This is not helpful, as there are times when this level of storage is exactly what you want. This paragraph reads as a "Just don't do this," paragraph rather than a "You can do this, here's why you'd want to and what pitfalls to avoid," paragraph. I've seen the same trick in exUserFolder (although I did not understand why it did that until yesterday). Also, the dev guide makes no mention of _v_ attributes be thread/connection specific, which is a dangerous oversight, IMHO. I've made the appropriate comments to the dev guide online. Thanks for everyone's help. Charlie.
-----Original Message----- From: kapil thangavelu [mailto:kthangavelu@earthlink.net] Sent: Thursday, August 15, 2002 7:43 PM To: Charlie Reiman; zope@zope.org Subject: Re: [Zope] Persistence Problem
as andy mentioned, _v_ volatile attributes are specific to a thread (well, technically there specific to the underlying zodb connection). they remain only as long as the object remains in memory/cache.
.......
I forget the geneology of this thread, but what was wrong with storing the data in a session? ----- Original Message ----- From: "Charlie Reiman" <creiman@kefta.com> To: <kthangavelu@earthlink.net>; <zope@zope.org> Sent: Friday, August 16, 2002 12:34 PM Subject: RE: [Zope] Persistence Problem
Thanks everyone. This dawned on me as I was on the train riding home. I also found __setstate__ and __getstate__ if I want to mess with the loading hooks (I don't).
http://www.zopelabs.com/cookbook/1002251996
The Zope Developer's guide does talk about module variables. But all it says is:
"Objects stored in modules but not in the ZODB are not persistent and not-thread safe. In general it's not a good idea to store data (as opposed to functions, and class definitions) in modules when using ZODB."
This is not helpful, as there are times when this level of storage is exactly what you want. This paragraph reads as a "Just don't do this," paragraph rather than a "You can do this, here's why you'd want to and what pitfalls to avoid," paragraph. I've seen the same trick in exUserFolder (although I did not understand why it did that until yesterday).
Also, the dev guide makes no mention of _v_ attributes be thread/connection specific, which is a dangerous oversight, IMHO.
I've made the appropriate comments to the dev guide online. Thanks for everyone's help.
Charlie.
-----Original Message----- From: kapil thangavelu [mailto:kthangavelu@earthlink.net] Sent: Thursday, August 15, 2002 7:43 PM To: Charlie Reiman; zope@zope.org Subject: Re: [Zope] Persistence Problem
as andy mentioned, _v_ volatile attributes are specific to a thread (well, technically there specific to the underlying zodb connection). they remain only as long as the object remains in memory/cache.
.......
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
I'm storing database connections, not simple objects (not even Zope objects). My DA has to store them, or at least have easy access to them. If you want the longer story... I'm implementing user keyed connections with exUserFolder and DCOracle2. The way it works is a user signs on with a id and password, which exUserFolder will then use to attempt to hook up to a database via DCO2. If it succeed, the user is considered authenticated and the connection is cached under DCO2 for future use. All future DCO2 transactions for that user funnel through her private connection. I know Zope DAs are not designed to do this (trust me, no one knows this more than me right now). It's way too complicated and an evolutionary dead end but my boss really wants it, so he gets it.
-----Original Message----- From: Chris McDonough [mailto:chrism@zope.com] Sent: Friday, August 16, 2002 9:36 AM To: Charlie Reiman; kthangavelu@earthlink.net; zope@zope.org Subject: Re: [Zope] Persistence Problem
I forget the geneology of this thread, but what was wrong with storing the data in a session?
----- Original Message ----- From: "Charlie Reiman" <creiman@kefta.com> To: <kthangavelu@earthlink.net>; <zope@zope.org> Sent: Friday, August 16, 2002 12:34 PM Subject: RE: [Zope] Persistence Problem
Thanks everyone. This dawned on me as I was on the train riding home. I also found __setstate__ and __getstate__ if I want to mess with the loading hooks (I don't).
http://www.zopelabs.com/cookbook/1002251996
The Zope Developer's guide does talk about module variables. But all it says is:
"Objects stored in modules but not in the ZODB are not persistent and not-thread safe. In general it's not a good idea to store data (as opposed to functions, and class definitions) in modules when using ZODB."
This is not helpful, as there are times when this level of storage is exactly what you want. This paragraph reads as a "Just don't do this," paragraph rather than a "You can do this, here's why you'd want to and what pitfalls to avoid," paragraph. I've seen the same trick in exUserFolder (although I did not understand why it did that until yesterday).
Also, the dev guide makes no mention of _v_ attributes be thread/connection specific, which is a dangerous oversight, IMHO.
I've made the appropriate comments to the dev guide online. Thanks for everyone's help.
Charlie.
-----Original Message----- From: kapil thangavelu [mailto:kthangavelu@earthlink.net] Sent: Thursday, August 15, 2002 7:43 PM To: Charlie Reiman; zope@zope.org Subject: Re: [Zope] Persistence Problem
as andy mentioned, _v_ volatile attributes are specific to a thread (well, technically there specific to the underlying zodb connection). they remain only as long as the object remains in memory/cache.
.......
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
On Friday 16 August 2002 09:35 am, Chris McDonough wrote:
I forget the geneology of this thread, but what was wrong with storing the data in a session?
they were database connections, (for others benefit) and hence not valid for being session stored due to pickle rules. -k
Charlie Reiman writes:
Thanks everyone. This dawned on me as I was on the train riding home. I also found __setstate__ and __getstate__ if I want to mess with the loading hooks (I don't).
http://www.zopelabs.com/cookbook/1002251996
The Zope Developer's guide does talk about module variables. But all it says is:
"Objects stored in modules but not in the ZODB are not persistent and not-thread safe. In general it's not a good idea to store data (as opposed to functions, and class definitions) in modules when using ZODB." You might look at "SharableResource" at
<http://www.dieter.handshake.de/pyprojects/zope> It has been developed for Zope 2.1.6, thus it may not run in modern Zope version. Dieter
participants (5)
-
Andy McKay -
Charlie Reiman -
Chris McDonough -
Dieter Maurer -
kapil thangavelu