Hi all, I'm finally revisiting the nasty part of my code that periodically hoses itself and fails to deliver resized images. ;-D I need to implement a cache of image objects in Zope (i.e. not in front of Zope), and it was suggested to me that the best way might be to use "Temporary Folders" in Zope 2.5.1. Anyway, I now see that Temporary Folder is a bit more magical than most Zope objects (not too surprizing I suppose). Anyway, what I'm trying to do is inherit from both Image and Temporary Folder and create the resized images in the object (supposing that they will then be stored temporarily). Perhaps this is naive, but it was my first guess as to the most logical way to do this. What I actually have looks like this: class VarImage(OFS.Image.Image, MountedTemporaryFolder) : """ Variable-size image object for Zope. """ #[... lots of hopefully irrelevant detail omitted ...] def _read_url_cmd(self, keystr): # Actual command recognition omitted for brevity, # but for example's sake: if keystr=='s100': return (100, 100) else return None def __getattr__(self, keystr): dimensions = self._read_url_cmd(keystr) if dimensions == None: # Unrecognized URL requested, pass through: # raise ValueError, 'Unrecognized VarImage scaling command.' try: return MountedTemporaryFolder.__getattr__(self, keystr) except: return OFS.Image.Image.__getattr__(self, keystr) else: nx, ny = dimensions # This part is messy, because I'm testing, eventually it # will do a bit more to try to get a cache hit instead of # creating the image everytime: threading.Lock() img_file, img_type = img_resize(StringIO(self.data), self.content_type, nx, ny) img = OFS.Image.Image(keystr, title='', file=img_file, content_type=img_type) self._setObject(keystr, img) # ^^^^^^-- This chokes #self.manage_addProduct['OFSP'].manage_addImage(keystr, '', # file=img_file, content_type=img_type) # ^^^^^^-- So does this when you uncomment it threading.Unlock() # Okay, now throw all that away and return self anyway. # just for debugging of course -- this is so I'm not debugging # the actual resizing code. return self Looking in Products/TemporaryFolder.py makes me think I'm being naive, though. I note that the meta_type for MountedTemporaryFolder says it's "Broken Temporary Folder". So where does "Temporary Folder" come from? It's here in the __init__.py: def initialize(context): import TemporaryFolder context.registerClass( TemporaryFolder.MountedTemporaryFolder, permission=TemporaryFolder.ADD_TEMPORARY_FOLDER_PERM, icon='www/tempfolder.gif', meta_type='Temporary Folder', constructors=(TemporaryFolder.constructTemporaryFolderForm, TemporaryFolder.constructTemporaryFolder) ) (constructTemporaryFolder doesn't appear to be particularly different from the more common "manage_add" function, except for the name, BTW). This appears to be doing something clever, but I don't get it. I can't figure out when and how the Temporary Folder acquires Folder-like behavior. Questions: 1. *Can* I inherit from Temporary Folder as desired, and if so, will it act as a Temporary Folder? 2. If so, how? Which object do I inherit from, and/or do I have to do anything special to make it non-broken, since it apparently starts out broken? Any clarifications much appreciated, thank you! Terry -- ------------------------------------------------------ Terry Hancock hancock@anansispaceworks.com Anansi Spaceworks http://www.anansispaceworks.com P.O. Box 60583 Pasadena, CA 91116-6583 ------------------------------------------------------
Terry Hancock wrote:
Hi all,
I'm finally revisiting the nasty part of my code that periodically hoses itself and fails to deliver resized images. ;-D
As a small idea, rather than inheritting from Image and TemporaryFolder, why not subclass Image (but only if you need to change its behaviour ;-) and then store you images in the Temporary Folder in the root of your ZODB? cheers, Chris
Even better, maybe just use RAM Cache Manager and give each resized image its own URL? On Sun, 2002-08-11 at 09:31, Chris Withers wrote:
Terry Hancock wrote:
Hi all,
I'm finally revisiting the nasty part of my code that periodically hoses itself and fails to deliver resized images. ;-D
As a small idea, rather than inheritting from Image and TemporaryFolder, why not subclass Image (but only if you need to change its behaviour ;-) and then store you images in the Temporary Folder in the root of your ZODB?
cheers,
Chris
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Chris McDonough wrote:
Even better, maybe just use RAM Cache Manager and give each resized image its own URL?
On Sun, 2002-08-11 at 09:31, Chris Withers wrote:
Terry Hancock wrote:
Hi all,
I'm finally revisiting the nasty part of my code that periodically hoses itself and fails to deliver resized images. ;-D
As a small idea, rather than inheritting from Image and TemporaryFolder, why not subclass Image (but only if you need to change its behaviour ;-) and then store you images in the Temporary Folder in the root of your ZODB?
Naturally I need to change Image's behavior, and indeed every image does get its own URL (this improves browser caching as well as any upstream caching). But for the system to essentially fail completely without a RAM cache manager is poor design, IMHO, and puts unnecessary constraints on the installation, when the product should really take care of this on its own. Likewise, using a centralized temporary folder puts complexity into the ZMI, when it shouldn't be there. I would then have to create a scheme of URLs for the temp storage distinct from the original location of the images. That's going to be very ugly. It's better to have the caching going on essentially at the source object. I considered and rejected both of those designs for what I consider to be good reasons. VarImage is a utility class and gets used for several different things in several places. To dump the complexity up a layer to the calling code just doesn't make sense. I *had* considered making the temp folder *contained* by the Image and not a superclass of it. But I'd still like to understand how and why a Temp folder comes into being. The source is not immediately transparent on this point. Why did it need to be different from other Zope objects? Why is there no such thing as a "Temporary Folder" class in Zope? I thought we liked object oriented programming. ;-D Now all of a sudden we abandon it. There must be a reason for that. It seems to me that it's impossible to subclass Temporary Folder, because there's no such class. I'd like to think that's wrong. Thanks, Terry -- ------------------------------------------------------ Terry Hancock hancock@anansispaceworks.com Anansi Spaceworks http://www.anansispaceworks.com P.O. Box 60583 Pasadena, CA 91116-6583 ------------------------------------------------------
There is a TemporaryFolder class in Zope. It's in lib/python/Products/TemporaryFolder.py and its named MountedTemporaryFolder. I recommend against subclassing MountedTemporaryFolder unless you want to spend the time to understand ZODB "mounting". A temporary folder is an object that is a mount point. It mounts a TemporaryStorage-backed database and masquerades as its root object. When you traverse one of these things, the __of__ method of the mount point object is called, and that returns the a Folder object that actually lives in another ZODB. To understand it fully, you'd need to read the source of TemporaryFolder.py and MountPoint.py. It's not always the most obvious of code, but it's not too terrible. But I honestly don't think using a TemporaryFolder for this is the right thing here. Your object should return different images when different methods are called on it, and each method should have its own URL. Get that working first, then integrate caching into it as an add-on service. You can try RAM Cache Manager or simple module-level globals keyed on the Zope physical path of your objects. This is not dumping the complexity up a level, it's just using existing services to speed things up (it won't "fail completely", your system will work without caching, just not as fast). On Sun, 2002-08-11 at 18:17, Terry Hancock wrote:
Chris McDonough wrote:
Even better, maybe just use RAM Cache Manager and give each resized image its own URL?
On Sun, 2002-08-11 at 09:31, Chris Withers wrote:
Terry Hancock wrote:
Hi all,
I'm finally revisiting the nasty part of my code that periodically hoses itself and fails to deliver resized images. ;-D
As a small idea, rather than inheritting from Image and TemporaryFolder, why not subclass Image (but only if you need to change its behaviour ;-) and then store you images in the Temporary Folder in the root of your ZODB?
Naturally I need to change Image's behavior, and indeed every image does get its own URL (this improves browser caching as well as any upstream caching). But for the system to essentially fail completely without a RAM cache manager is poor design, IMHO, and puts unnecessary constraints on the installation, when the product should really take care of this on its own.
Likewise, using a centralized temporary folder puts complexity into the ZMI, when it shouldn't be there. I would then have to create a scheme of URLs for the temp storage distinct from the original location of the images. That's going to be very ugly. It's better to have the caching going on essentially at the source object.
I considered and rejected both of those designs for what I consider to be good reasons. VarImage is a utility class and gets used for several different things in several places. To dump the complexity up a layer to the calling code just doesn't make sense.
I *had* considered making the temp folder *contained* by the Image and not a superclass of it. But I'd still like to understand how and why a Temp folder comes into being. The source is not immediately transparent on this point. Why did it need to be different from other Zope objects? Why is there no such thing as a "Temporary Folder" class in Zope? I thought we liked object oriented programming. ;-D Now all of a sudden we abandon it. There must be a reason for that.
It seems to me that it's impossible to subclass Temporary Folder, because there's no such class. I'd like to think that's wrong.
Thanks, Terry
-- ------------------------------------------------------ Terry Hancock hancock@anansispaceworks.com Anansi Spaceworks http://www.anansispaceworks.com P.O. Box 60583 Pasadena, CA 91116-6583 ------------------------------------------------------
Chris McDonough wrote:
There is a TemporaryFolder class in Zope. It's in lib/python/Products/TemporaryFolder.py and its named MountedTemporaryFolder. I recommend against subclassing MountedTemporaryFolder unless you want to spend the time to understand ZODB "mounting".
Actually, I got it working by inheriting from ObjectManager and simply *containing* a Temporary Folder. I'm not sure if that was strictly necessary, but I like it better this way anyway (for one thing, it makes it possible to query the cache independently of my __getattr__ overload which is one of those "kids don't try this at home" techniques ;-D, so its good to minimize the impact as much as possible).
A temporary folder is an object that is a mount point. It mounts a TemporaryStorage-backed database and masquerades as its root object. When you traverse one of these things, the __of__ method of the mount point object is called, and that returns the a Folder object that actually lives in another ZODB. To understand it fully, you'd need to read the source of TemporaryFolder.py and MountPoint.py.
Chris, I know "beggars can't be choosers", but seriously, I think it would be a big improvement of the code to put the above text into a nice sweet little comment block inside TemporaryFolder.py. The existing comments are inadequate (this is an amazing understatement). It will do exactly as you wrote it. <rant> Yes, it's open-source. Yes, no one is paying you for it. So what? You released it because you want it to be used and improved. You want people to help you with it. You want people to document it for you (also for free, I might add). And you don't really want to waste your time answering the same questions over and over again. So why are you shooting yourself in the foot by making it difficult for them? Comments are cheap, and a real bargain in the long run. </rant>
But I honestly don't think using a TemporaryFolder for this is the right thing here. Your object should return different images when different methods are called on it, and each method should have its own URL.
Each image *does* get a separate URL. Downstream caches can take advantage of this. And yes, that's why I did it that way. Anyway I disagree on the design decision. No offense, but I have to live with it, so I'll gamble on my own judgement. Anyway -- thanks for the description of how TemporaryFolder is meant to work. And I guess you must be the author of the module itself, in which case, thanks for that too, it's a nice addition to Zope. VarImage now passes my stress test, which is to ask for a page with 300 different sizes of the same image (this mainly tests the conflict problem that arises when a single storage location is used). Cache performance is good -- the second request for the same page is very quick indeed (this tests the scaling performance). So I'm a happy camper. We'll have a copy on our project site at Sourceforge sometime soon, if anyone would like to use a freely-resizing image type in Zope, we've found it quite useful in the design process: http://sourceforge.net/projects/spacelift (I separated VarImage out into a separate product for easier testing and because it's probably useful for more than just Narya). Thanks, Terry -- ------------------------------------------------------ Terry Hancock hancock@anansispaceworks.com Anansi Spaceworks http://www.anansispaceworks.com P.O. Box 60583 Pasadena, CA 91116-6583 ------------------------------------------------------
Chris, I know "beggars can't be choosers", but seriously, I think it would be a big improvement of the code to put the above text into a nice sweet little comment block inside TemporaryFolder.py. The existing comments are inadequate (this is an amazing understatement). It will do exactly as you wrote it.
<rant> Yes, it's open-source. Yes, no one is paying you for it. So what? You released it because you want it to be used and improved. You want people to help you with it. You want people to document it for you (also for free, I might add). And you don't really want to waste your time answering the same questions over and over again. So why are you shooting yourself in the foot by making it difficult for them? Comments are cheap, and a real bargain in the long run. </rant>
Done.
Anyway I disagree on the design decision. No offense, but I have to live with it, so I'll gamble on my own judgement.
Sure.
Anyway -- thanks for the description of how TemporaryFolder is meant to work. And I guess you must be the author of the module itself, in which case, thanks for that too, it's a nice addition to Zope.
No problem, glad it was useful. - C
participants (3)
-
Chris McDonough -
Chris Withers -
Terry Hancock