Photo product: how to recreate thumbnails with Imagemagick
Hi all, I'm using Ron's great Photo on my site. Until recently I was unable to use Imagemagick and had to use PIL to render the different images. I've got Imagemagick working now and the difference is dramatic! The PIL-rendered versions aren't even close to the Imagemagick-rendered ones. Now the question, can I force Zope to create new version of these thumbnails using Imagemagick without recreating all of them from scratch? -Tim -- Tim Wilson | Visit Sibley online: | Check out: Henry Sibley HS | http://www.isd197.org | http://www.zope.com W. St. Paul, MN | | http://slashdot.org wilson@visi.com | <dtml-var pithy_quote> | http://linux.com
-----Original Message----- From: zope-admin@zope.org [mailto:zope-admin@zope.org]On Behalf Of Timothy Wilson
Now the question, can I force Zope to create new version of these thumbnails using Imagemagick without recreating all of them from scratch?
If you don't have too many Photos, you can go into each one and change the engine to 'ImageMagick' and then go into the displays and select "Regenerate All". If you have too many for that, you could create a Python Script to iterate through them, calling manage_changeProperties() to change the engine and manage_regenDisplays() to regenerate them. If you're using the latest version, that might go something like this (untested): for photo in context.objectValues(['Photo']): photo.propertysheets.get('photoconf').manage_changeProperties(engine='ImageM agick') photo.manage_regenDisplays() This may take a while to run if you have a lot of images, but as you've seen, it's worth it. In fact, I've made ImageMagick the default engine in the latest version because of its superior quality. Enjoy! _______________________ Ron Bickers Logic Etc, Inc.
On Thu, 25 Oct 2001, Ron Bickers wrote:
If you don't have too many Photos, you can go into each one and change the engine to 'ImageMagick' and then go into the displays and select "Regenerate All".
Yes, I noticed that after posting the question. Doh! Unfortunately, when I look at the edit tab of one of the old Photo instances it says "No value for image_engines ." next to the Engine property. If I try to "Regenerate all" I get the following: Error Type: AttributeError Error Value: 'None' object has no attribute 'getProperty' The old PIL-generated images still display, but something seems wrong here. -Tim -- Tim Wilson | Visit Sibley online: | Check out: Henry Sibley HS | http://www.isd197.org | http://www.zope.com W. St. Paul, MN | | http://slashdot.org wilson@visi.com | <dtml-var pithy_quote> | http://linux.com
-----Original Message----- From: Timothy Wilson [mailto:wilson@visi.com] Sent: Thursday, October 25, 2001 11:34 PM
Unfortunately, when I look at the edit tab of one of the old Photo instances it says "No value for image_engines ." next to the Engine property.
If you just upgraded to 1.2.0 from a prior version, you need to run 'upgrade' on each of your Photo Folder objects. As in, http://myhost/myPhotoFolder/upgrade. _______________________ Ron Bickers Logic Etc, Inc.
On Thu, 25 Oct 2001, Ron Bickers wrote:
Unfortunately, when I look at the edit tab of one of the old Photo instances it says "No value for image_engines ." next to the Engine property.
If you just upgraded to 1.2.0 from a prior version, you need to run 'upgrade' on each of your Photo Folder objects. As in, http://myhost/myPhotoFolder/upgrade.
Hi Ron, Thanks for the reply. Unfortunately, though calling the 'upgrade' method reports "Done.", nothing seems to have changed. Same error message. -Tim -- Tim Wilson | Visit Sibley online: | Check out: Henry Sibley HS | http://www.isd197.org | http://www.zope.com W. St. Paul, MN | | http://slashdot.org wilson@visi.com | <dtml-var pithy_quote> | http://linux.com
-----Original Message----- From: Timothy Wilson [mailto:wilson@visi.com]
If you just upgraded to 1.2.0 from a prior version, you need to run 'upgrade' on each of your Photo Folder objects. As in, http://myhost/myPhotoFolder/upgrade.
Hi Ron,
Thanks for the reply. Unfortunately, though calling the 'upgrade' method reports "Done.", nothing seems to have changed. Same error message.
The property is still there but broken, or the regenerate all error message still shows? Which version did you upgrade from? Starting with 1.2.0, you change the engine on the Settings tab. Even if the old property is broken, it should still work. Can you send the traceback of the error? _______________________ Ron Bickers Logic Etc, Inc.
On Fri, 26 Oct 2001, Ron Bickers wrote:
Starting with 1.2.0, you change the engine on the Settings tab. Even if the old property is broken, it should still work.
Can you send the traceback of the error?
Yeah, I tried chaning it from the Settings tab. That's where I got the error. Traceback follows: <!-- Traceback (innermost last): File /home/wilson/zope/2.3.3/lib/python/ZPublisher/Publish.py, line 223, in publish_module File /home/wilson/zope/2.3.3/lib/python/ZPublisher/Publish.py, line 187, in publish File /home/wilson/zope/2.3.3/lib/python/Zope/__init__.py, line 221, in zpublisher_exception_hook (Object: ElementWithAttributes) File /home/wilson/zope/2.3.3/lib/python/ZPublisher/Publish.py, line 171, in publish File /home/wilson/zope/2.3.3/lib/python/ZPublisher/mapply.py, line 160, in mapply (Object: manage_editSettings) File /home/wilson/zope/2.3.3/lib/python/ZPublisher/Publish.py, line 112, in call_object (Object: manage_editSettings) File /usr/local/zope/Products/Photo/Photo.py, line 455, in manage_editSettings (Object: ElementWithAttributes) AttributeError: (see above) --> -Tim -- Tim Wilson | Visit Sibley online: | Check out: Henry Sibley HS | http://www.isd197.org | http://www.zope.com W. St. Paul, MN | | http://slashdot.org wilson@visi.com | <dtml-var pithy_quote> | http://linux.com
Hi everyone, If you'd like to see some images rendered with PIL and ImageMagick, have a look at http://www.isd197.org/sibley/potd/200110/. See if you can guess which day I switched to using ImageMagick. (Hint: It's toward the end.) I think it's pretty obvious. -Tim -- Tim Wilson | Visit Sibley online: | Check out: Henry Sibley HS | http://www.isd197.org | http://www.zope.com W. St. Paul, MN | | http://slashdot.org wilson@visi.com | <dtml-var pithy_quote> | http://linux.com
On Thu, Oct 25, 2001 at 06:07:01PM -0400, Ron Bickers wrote:
This may take a while to run if you have a lot of images, but as you've seen, it's worth it. In fact, I've made ImageMagick the default engine in the latest version because of its superior quality.
Does it run faster or slower than PIL? By about how much? -- paul winkler home: http://www.slinkp.com music: http://www.reacharms.com calendars: http://www.calendargalaxy.com
On Fri, 26 Oct 2001, Paul Winkler wrote:
On Thu, Oct 25, 2001 at 06:07:01PM -0400, Ron Bickers wrote:
This may take a while to run if you have a lot of images, but as you've seen, it's worth it. In fact, I've made ImageMagick the default engine in the latest version because of its superior quality.
Does it run faster or slower than PIL? By about how much?
If I understand the product correctly, the Photo product works by rendering all the various sizes at the time you create the Photo instance. Therefore, once you've created the 'Photo' the program used to create the images should have no bearing on how fast the images appear on your Web site. In other words, Photo doesn't create the thumbnails on the fly. In my experience with 1024x768 images, the difference in speed between ImageMagick and PIL is unnoticeable. I've got a pretty fast machine though. -Tim -- Tim Wilson | Visit Sibley online: | Check out: Henry Sibley HS | http://www.isd197.org | http://www.zope.com W. St. Paul, MN | | http://slashdot.org wilson@visi.com | <dtml-var pithy_quote> | http://linux.com
-----Original Message----- From: zope-admin@zope.org [mailto:zope-admin@zope.org]On Behalf Of Timothy Wilson
Does it run faster or slower than PIL? By about how much?
If I understand the product correctly, the Photo product works by rendering all the various sizes at the time you create the Photo instance. Therefore, once you've created the 'Photo' the program used to create the images should have no bearing on how fast the images appear on your Web site. In other words, Photo doesn't create the thumbnails on the fly.
That's correct. Both engines are too slow for on the fly rendering in most cases, IMO. The engine makes a different in creation speed and image quality, but once the images are created, there's no difference in speed.
In my experience with 1024x768 images, the difference in speed between ImageMagick and PIL is unnoticeable. I've got a pretty fast machine though.
My development machine is a PIII 550 RedHat box, which does show a noticeable difference between the two. I don't remember the exact numbers, but it was something like 4 or 5 seconds for PIL to render the default displays on a 330k image, and nearly 20 seconds for ImageMagick. It may vary greatly depending on the image size and complexity. My PIII 1GHz shows much less of a difference in the two, but I can still tell if I count. In any case, I think the creation speed difference of ImageMagick is well worth the better quality. _______________________ Ron Bickers Logic Etc, Inc.
I wrote:
Does it run faster or slower than PIL? By about how much?
... and got two contradictory replies, so clearly I'll have to check it out myself. :)
If I understand the product correctly, the Photo product works by rendering all the various sizes at the time you create the Photo instance. Therefore, once you've created the 'Photo' the program used to create the images should have no bearing on how fast the images appear on your Web site. In other words, Photo doesn't create the thumbnails on the fly.
Right, but I'm building a site where users will upload some pretty large images, and how long they have to wait for their first view of the thumbnails may be more important than how nice they look in this case. But with any luck, both will prove to be "fast enough" under load. -- paul winkler home: http://www.slinkp.com music: http://www.reacharms.com calendars: http://www.calendargalaxy.com
-----Original Message----- From: zope-admin@zope.org [mailto:zope-admin@zope.org]On Behalf Of Paul Winkler
Does it run faster or slower than PIL? By about how much?
... and got two contradictory replies, so clearly I'll have to check it out myself. :)
Good idea. It really does vary quite a bit depending on hardware and the photo itself.
Right, but I'm building a site where users will upload some pretty large images, and how long they have to wait for their first view of the thumbnails may be more important than how nice they look in this case. But with any luck, both will prove to be "fast enough" under load.
It's pretty sad that an initial wait time would be more important than quality. I guess that's the American way. ;-) Seriously, though, I suppose if they aren't going to be viewed many times after the upload, it makes sense. If you have a fast server, either should be okay. Let me know what you experience. _______________________ Ron Bickers Logic Etc, Inc.
From: Timothy Wilson <wilson@visi.com>
If I understand the product correctly, the Photo product works by rendering all the various sizes at the time you create the Photo instance. Therefore, once you've created the 'Photo' the program used to create the images should have no bearing on how fast the images appear on your Web site. In other words, Photo doesn't create the thumbnails on the fly.
Right... that's fine for some types of sites. Others would benefit more from cached on-the-fly generation.
In my experience with 1024x768 images, the difference in speed between ImageMagick and PIL is unnoticeable. I've got a pretty fast machine though.
Well, the thing to do then is benchmark 1000 or 10000 or how ever many it becomes not only noticeable, but significant enough to swamp extraneous first-order overheads, so you can get the relative performance....
-----Original Message----- From: zope-admin@zope.org [mailto:zope-admin@zope.org]On Behalf Of marc lindahl
Right... that's fine for some types of sites. Others would benefit more from cached on-the-fly generation.
Can you provide an example application where this might be useful? Also, how would you propose caching on-the-fly photos? As stated in the RAM Cache documentation: "Although Zope does not prevent you from doing so, it generally does not make sense to associate an image or a file object with a RAM cache manager. It will not cache the image or file data, since the data is already available in RAM. However, another kind of cache manager, an accelerated HTTP cache manager, is available and is suitable for images and file objects." So it seems it wouldn't make sense to use a RAM cache with the generated Photo displays, and the accelerated cache is for setting headers for browsers and proxies, etc. to cache. A RAM cache might be useful for on-the-fly photo caching, but wouldn't that quickly eat up memory? I suppose I could store them just like the other display sizes and associate a timeout with them. I'm not sure if that would be sufficient or not. Much of this might depend on the application. Should there be infinite size options for the user such that you could end up caching dozens or hundreds of display sizes for a given photo? Thoughts? _______________________ Ron Bickers Logic Etc, Inc.
From: "Ron Bickers" <rbickers-dated-1004726577.71eff0@logicetc.com>
Can you provide an example application where this might be useful?
What about something where the user can select levels of 'zoom', perhaps out of 10 or 20 possibilities? If there are lots and lots of images on the site, then the probability is that few of them will have all zoom levels used. But it also means that if all the zooms are stored with in ZODB you'll get some nice bloat, or if they're in the external fs, you have a big mess that someone has to back up. So it makes sense to generate them on the fly, and cache them so that popular ones are re-used, cutting down CPU processing.
Also, how would you propose caching on-the-fly photos?
I wish I knew! I'm trying to figure that out, but there's little documentation on the RAM Cache manager or howto's or anything...
As stated in the RAM Cache documentation:
"Although Zope does not prevent you from doing so, it generally does not make sense to associate an image or a file object with a RAM cache manager. It will not cache the image or file data, since the data is already available in RAM. However, another kind of cache manager, an accelerated HTTP cache manager, is available and is suitable for images and file objects."
They're referring to Image, which is of course computation-free. They do suggest that it be used for something with 'expensive' computation, like a database lookup. For many applications, using IM or something for resizing or munging an image would qualify as 'expensive'.
So it seems it wouldn't make sense to use a RAM cache with the generated Photo displays, and the accelerated cache is for setting headers for browsers and proxies, etc. to cache. A RAM cache might be useful for on-the-fly photo caching, but wouldn't that quickly eat up memory? I suppose I could store them just like the other display sizes and associate a timeout with them. I'm not sure if that would be sufficient or not.
I was thinking that you would have one RAM Cache for the site, not one for each image. So you can control the overall amount of memory. And, it does what a cache is supposed to, which is save the most amount of repetitive re-computation of the same thing (that is, the most popular image sizes).
Much of this might depend on the application. Should there be infinite size options for the user such that you could end up caching dozens or hundreds of display sizes for a given photo?
For somethings like a photo album, you have only one re-size, which is the thumbnail. Or maybe two, if you want a view that's 100% screensize or something. But the 'zoom' example above is a useful one for multiple resizing. In any case, cached or stored, will benefit from limiting the user's options. Because, this provides the opportunity for overlap among multiple users, which benefits any type of caching (after all, storing the thumbs is still caching...)
-----Original Message----- From: marc lindahl [mailto:marc@bowery.com] Sent: Saturday, October 27, 2001 1:26 AM
What about something where the user can select levels of 'zoom', perhaps out of 10 or 20 possibilities?
Well, the default photo object provides 5 "zooms" as it is. You could define 10 or 20, but as you point out, that would indeed bloat the ZODB or file system. Do you think it's still a reasonable thing to require the developer to preset display sizes, no matter how many and whether or not they're rendered beforehand or on-the-fly?
They're referring to Image, which is of course computation-free. They do suggest that it be used for something with 'expensive' computation, like a database lookup. For many applications, using IM or something for resizing or munging an image would qualify as 'expensive'.
Indeed, it is very expensive. In fact, so expensive in some cases that I hesitate to make it an option even with caching, but I can see how it may be beneficial. The biggest problem I see with using a RAM cache with rendered photos is that many photos of a moderate size could eat a lot of RAM. Take 50 different photos where three rendered sizes average 50k each. Cache them and you've eaten 7.5MB of RAM. My personal site currently has 414 photos with a couple more hundred on the way. If someone went through them all at a single display size, and you RAM cached that one size of each photo at 50k, that's an enormous belly filled with happy memories...nearly 21MB of happy memories. Obviously you would want to put a cap on the RAM cache size, but then you greatly increase the occurrence of the expensive rendering and thus the practical effectiveness of the cache. Am I missing something?
I was thinking that you would have one RAM Cache for the site, not one for each image. So you can control the overall amount of memory. And, it does what a cache is supposed to, which is save the most amount of repetitive re-computation of the same thing (that is, the most popular image sizes).
I was suggesting that instead of using a RAM cache at all, that the displays be created on-the-fly and stored like they are now (in the ZODB or ExtImage). I could then add a timeout property for displays such that when a photo is requested, if the timeout passed for a given display size, that display size would be purged. Would that cause too much write activity for an undoable database? If you set the timeout to a really large number, much larger than a "normal" cache might be, I don't think it would purged less often. I think you'd still end up with only the most popular display sizes taking up space. That is the goal, right? ... having a large number of possible display sizes without them eating ZODB/file system space? Do you see any other problems with this idea? I actually like the sound of it, but I could be overlooking some issues.
For somethings like a photo album, you have only one re-size, which is the thumbnail.
I would consider my site to be a photo album, and I have 5 displays + a thumbnail for each photo. Most people using the Photo product will likely have the same since that's the default.
In any case, cached or stored, will benefit from limiting the user's options. Because, this provides the opportunity for overlap among multiple users, which benefits any type of caching (after all, storing the thumbs is still caching...)
I don't understand what mean. Can you clarify? _______________________ Ron Bickers Logic Etc, Inc.
I am using Photo in a product which lets the user lump together text an photos to create a story with pictures. I have no displays created automatically. However the user can define a size of the picture she wants to use. From this I then create a display. Since the user will check here "story" while working on it the displays are created at a very early state in the game where waiting is tolerable. The idea of timing out displays is appealing. Robert ----- Original Message ----- From: "Ron Bickers" <rbickers-dated-1004769126.c039d8@logicetc.com> To: "marc lindahl" <marc@bowery.com> Cc: "Zope listserv" <zope@zope.org> Sent: Saturday, October 27, 2001 8:32 AM Subject: RE: [Zope] Caching on-the-fly resized photos
-----Original Message----- From: marc lindahl [mailto:marc@bowery.com] Sent: Saturday, October 27, 2001 1:26 AM
What about something where the user can select levels of 'zoom', perhaps out of 10 or 20 possibilities?
Well, the default photo object provides 5 "zooms" as it is. You could define 10 or 20, but as you point out, that would indeed bloat the ZODB or file system.
Do you think it's still a reasonable thing to require the developer to preset display sizes, no matter how many and whether or not they're rendered beforehand or on-the-fly?
They're referring to Image, which is of course computation-free. They do suggest that it be used for something with 'expensive' computation, like a database lookup. For many applications, using IM or something for resizing or munging an image would qualify as 'expensive'.
Indeed, it is very expensive. In fact, so expensive in some cases that I hesitate to make it an option even with caching, but I can see how it may be beneficial.
The biggest problem I see with using a RAM cache with rendered photos is that many photos of a moderate size could eat a lot of RAM. Take 50 different photos where three rendered sizes average 50k each. Cache them and you've eaten 7.5MB of RAM. My personal site currently has 414 photos with a couple more hundred on the way. If someone went through them all at a single display size, and you RAM cached that one size of each photo at 50k, that's an enormous belly filled with happy memories...nearly 21MB of happy memories.
Obviously you would want to put a cap on the RAM cache size, but then you greatly increase the occurrence of the expensive rendering and thus the practical effectiveness of the cache. Am I missing something?
I was thinking that you would have one RAM Cache for the site, not one for each image. So you can control the overall amount of memory. And, it does what a cache is supposed to, which is save the most amount of repetitive re-computation of the same thing (that is, the most popular image sizes).
I was suggesting that instead of using a RAM cache at all, that the displays be created on-the-fly and stored like they are now (in the ZODB or ExtImage). I could then add a timeout property for displays such that when a photo is requested, if the timeout passed for a given display size, that display size would be purged. Would that cause too much write activity for an undoable database? If you set the timeout to a really large number, much larger than a "normal" cache might be, I don't think it would purged less often. I think you'd still end up with only the most popular display sizes taking up space. That is the goal, right? ... having a large number of possible display sizes without them eating ZODB/file system space?
Do you see any other problems with this idea? I actually like the sound of it, but I could be overlooking some issues.
For somethings like a photo album, you have only one re-size, which is the thumbnail.
I would consider my site to be a photo album, and I have 5 displays + a thumbnail for each photo. Most people using the Photo product will likely have the same since that's the default.
In any case, cached or stored, will benefit from limiting the user's options. Because, this provides the opportunity for overlap among multiple users, which benefits any type of caching (after all, storing the thumbs is still caching...)
I don't understand what mean. Can you clarify?
_______________________
Ron Bickers Logic Etc, Inc.
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
-----Original Message----- From: zope-admin@zope.org [mailto:zope-admin@zope.org]On Behalf Of Robert Rottermann
I have no displays created automatically. However the user can define a size of the picture she wants to use. From this I then create a display. Since the user will check here "story" while working on it the displays are created at a very early state in the game where waiting is tolerable.
The idea of timing out displays is appealing.
Ok. I've implemented something that seems like a good idea, but I'm not sure how it will fly in production. Also, I don't think it directly addresses the above scenario. I could probably extend what I have done to create displays (not just the images of predefined displays, but actual new display sizes) when a custom size is requested. I could then tag it such that when it times out the entire display (not just the stored image) is removed. Robert, would that cover what you're looking for? Read on and it should make more sense... The current CVS at sf.net/projects/zopephoto has options for on-the-fly generation of predefined display sizes and a timeout (in minutes) associated with each Photo. Each of these settings can be set in a Photo Folder to control the settings in all contained Photos. If the "pregenerate" option is chosen and timeout is set to zero, Photos behave exactly has they have to date; all displays are generated when the Photo is added/updated and the are never purged. If the pregenerate option is not chosen, displays will be generated on-the-fly as they are requested. The timeout is the tricky part. Ideally, every time a display is requested, an *access* time of that object would be updated and thus the age of the display would be reset to 0. This way, commonly accessed display sizes would never be purged. Since ZODB doesn't keep an access time, I could just use the mod time and update it when the display is requested. The problem with this is that it would cause a large number of writes to the ZODB, and since ZODB is append only, it would quickly bloat. Since I couldn't come up with a better way to track access time, I compromised. Every time a display is requested, the age is compared with the timeout. If the age is more than half the timeout, the display's mod time is reset to zero and any other display sizes whose age is greater than timeout are purged. With regular ZODB packs (which I assume most people do anyway), and high timeouts (like several days), this might actually work okay. In addition, you can run manage_purgeDisplays() or manage_cleanDisplays() which purges all displays (regardless of age) and purges all expired displays, respectively. Running these on the Photo Folder will process all contained Photos. There is a button in the management interface for each of these. I can see there might be the desire to have certain displays never expire (such as a thumbnail), but instead of adding extra features for that, I'll assume for now that in practice, thumbnails would be viewed often enough to not be purged anyway. We'll see. Also, manage_purgeDisplays() and manage_cleanDisplays() each take an 'exclude' parameter that is a tuple of displays to *not* purge/clean. In each case, excluded displays will have their age reset to zero. From the management interface, if you "check" a display(s) and then purge/clean, those displays will be excluded. This might seem backwards, but I think it's more practical. If you're going to try this out, remember to run the upgrade External Method as outlined in UPGRADE.txt. Any and all feedback and suggestions would be greatly appreciated. Thanks! _______________________ Ron Bickers Logic Etc, Inc.
Ron, you're a freekin genious!
Hi Ron, first thanks a lot for your efforts. I have installed the version. And did run update.py (got a Done) However I get an error when accessing the settings tab on a photo. Error value: name 'store' is not defined Source of /AA/xx/bilder/abhoren.jpg/manage_editSettingsForm with fault: ... <th class="form-label" align="right">Store:</th> <td> <dtml-if "store == 'Image'"> ... I probably have created that photo trough a product of mine. So may I could forego necessary settings??? I have two small wishes: - I believe you are mixing tabs and spaces to indent your code. On windows I have problems to edit it. I first have to load it into Wing and have it reformatted. It would be nice if you could make your editor to use spaces only. - please put the update.py in Photo.Extensions. Its easier to use (must not be copied). Thanks again Robert ----- Original Message ----- From: "Ron Bickers" <rbickers-dated-1005080052.5dd4ca@logicetc.com> To: "Robert Rottermann" <robert@redcor.ch>; "marc lindahl" <marc@bowery.com> Cc: "Zope listserv" <zope@zope.org> Sent: Tuesday, October 30, 2001 9:54 PM Subject: RE: [Zope] Caching on-the-fly resized photos (implemented)
-----Original Message----- From: zope-admin@zope.org [mailto:zope-admin@zope.org]On Behalf Of Robert Rottermann
I have no displays created automatically. However the user can define a size of the picture she wants to use. From this I then create a display. Since the user will check here "story" while working on it the displays are created at a very early state in the game where waiting is tolerable.
The idea of timing out displays is appealing.
Ok. I've implemented something that seems like a good idea, but I'm not sure how it will fly in production. Also, I don't think it directly addresses the above scenario. I could probably extend what I have done to create displays (not just the images of predefined displays, but actual new display sizes) when a custom size is requested. I could then tag it such that when it times out the entire display (not just the stored image) is removed. Robert, would that cover what you're looking for? Read on and it should make more sense...
The current CVS at sf.net/projects/zopephoto has options for on-the-fly generation of predefined display sizes and a timeout (in minutes) associated with each Photo. Each of these settings can be set in a Photo Folder to control the settings in all contained Photos.
If the "pregenerate" option is chosen and timeout is set to zero, Photos behave exactly has they have to date; all displays are generated when the Photo is added/updated and the are never purged. If the pregenerate option is not chosen, displays will be generated on-the-fly as they are requested.
The timeout is the tricky part. Ideally, every time a display is requested, an *access* time of that object would be updated and thus the age of the display would be reset to 0. This way, commonly accessed display sizes would never be purged. Since ZODB doesn't keep an access time, I could just use the mod time and update it when the display is requested. The problem with this is that it would cause a large number of writes to the ZODB, and since ZODB is append only, it would quickly bloat.
Since I couldn't come up with a better way to track access time, I compromised. Every time a display is requested, the age is compared with the timeout. If the age is more than half the timeout, the display's mod time is reset to zero and any other display sizes whose age is greater than timeout are purged. With regular ZODB packs (which I assume most people do anyway), and high timeouts (like several days), this might actually work okay.
In addition, you can run manage_purgeDisplays() or manage_cleanDisplays() which purges all displays (regardless of age) and purges all expired displays, respectively. Running these on the Photo Folder will process all contained Photos. There is a button in the management interface for each of these.
I can see there might be the desire to have certain displays never expire (such as a thumbnail), but instead of adding extra features for that, I'll assume for now that in practice, thumbnails would be viewed often enough to not be purged anyway. We'll see.
Also, manage_purgeDisplays() and manage_cleanDisplays() each take an 'exclude' parameter that is a tuple of displays to *not* purge/clean. In each case, excluded displays will have their age reset to zero. From the management interface, if you "check" a display(s) and then purge/clean, those displays will be excluded. This might seem backwards, but I think it's more practical.
If you're going to try this out, remember to run the upgrade External Method as outlined in UPGRADE.txt. Any and all feedback and suggestions would be greatly appreciated.
Thanks! _______________________
Ron Bickers Logic Etc, Inc.
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
-----Original Message----- From: Robert Rottermann [mailto:robert@redcor.ch]
Error value: name 'store' is not defined
Hmmm... Versions prior to 1.2.0 did not save the 'store' (because it can't be changed anyway), but I later decided it was worth showing. The upgrade *should* take care of that, but something must be amiss. From what version did you upgrade?
I have two small wishes: - I believe you are mixing tabs and spaces to indent your code.
Probably. Never bothered checking it out (bad me), but I didn't think it was doing that. I guess Python's good about letting me be lazy in that regard.
It would be nice if you could make your editor to use spaces only.
I'd prefer that too. Now I just have to figure out how. :-)
- please put the update.py in Photo.Extensions. Its easier to use (must not be copied).
Cool. I didn't know that. Done for the next version. Thanks for the feedback! _______________________ Ron Bickers Logic Etc, Inc.
Hi Ron, I reused update and now it works. Maybe I did not apply in in the Photofolder itself (I have lots of them). If you would create an message telling the number of changed photos instead of only Done such misuse could be better spotted. thank you Robert ----- Original Message ----- From: "Ron Bickers" <rbickers-dated-1005085034.c5868a@logicetc.com> To: "Robert Rottermann" <robert@redcor.ch> Cc: "Zope listserv" <zope@zope.org> Sent: Tuesday, October 30, 2001 11:17 PM Subject: RE: [Zope] Caching on-the-fly resized photos (implemented)
-----Original Message----- From: Robert Rottermann [mailto:robert@redcor.ch]
Error value: name 'store' is not defined
Hmmm... Versions prior to 1.2.0 did not save the 'store' (because it can't be changed anyway), but I later decided it was worth showing. The upgrade *should* take care of that, but something must be amiss. From what version did you upgrade?
I have two small wishes: - I believe you are mixing tabs and spaces to indent your code.
Probably. Never bothered checking it out (bad me), but I didn't think it was doing that. I guess Python's good about letting me be lazy in that regard.
It would be nice if you could make your editor to use spaces only.
I'd prefer that too. Now I just have to figure out how. :-)
- please put the update.py in Photo.Extensions. Its easier to use (must not be copied).
Cool. I didn't know that. Done for the next version.
Thanks for the feedback! _______________________
Ron Bickers Logic Etc, Inc.
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
-----Original Message----- From: Robert Rottermann [mailto:robert@redcor.ch]
I reused update and now it works. Maybe I did not apply in in the Photofolder itself (I have lots of them). If you would create an message telling the number of changed photos instead of only Done such misuse could be better spotted.
Ok. "return 'Done.'" is translated, "Well, I have to return *something*, then I have to get some sleep." I'll work on making it more informative. I'm glad it's working for you now. _______________________ Ron Bickers Logic Etc, Inc.
Hi, Ron Bickers wrote:
The timeout is the tricky part. Ideally, every time a display is requested, an *access* time of that object would be updated and thus the age of the display would be reset to 0. This way, commonly accessed display sizes would never be purged. Since ZODB doesn't keep an access time, I could just use the mod time and update it when the display is requested. The problem with this is that it would cause a large number of writes to the ZODB, and since ZODB is append only, it would quickly bloat.
Why don't you invent an "access time" attribute yourself, and just update it at every access? markus -- You don't have to be Microsoft to suck... but it helps. (Tim Hammerquist in comp.lang.python)
-----Original Message----- From: zope-admin@zope.org [mailto:zope-admin@zope.org]On Behalf Of Markus Schaber
Since ZODB doesn't keep an access time, I could just
use the mod time and update it when the display is requested. The problem with this is that it would cause a large number of writes to the ZODB, and since ZODB is append only, it would quickly bloat.
Why don't you invent an "access time" attribute yourself, and just update it at every access?
Updating any attribute (including my own access time attribute) on every access would cause Data.fs to bloat quickly, because updating an object appends a new version of it to the end of the file. This means that on every view of an image, Data.fs would grow by at least the size of that image. Also, the advantage of using the built in mod time is that the age is automatically reset any time the object is modified, so I don't have to keep up with it every place it might be updated. _______________________ Ron Bickers Logic Etc, Inc.
participants (16)
-
marc lindahl -
Markus Schaber -
Paul Winkler -
Robert Rottermann -
Ron Bickers -
Ron Bickers -
Ron Bickers -
Ron Bickers -
Ron Bickers -
Ron Bickers -
Ron Bickers -
Ron Bickers -
Ron Bickers -
Ron Bickers -
Ron Bickers -
Timothy Wilson