popen2 problem with Zope 2.7.0 and Photo
Greetings Zopistas. Since I've upgraded from 2.6.2 to 2.7.0, the following product code causes zope to hang: elif engine == 'ImageMagick': # Use ImageMagick if sys.platform == 'win32': from win32pipe import popen2 imgin, imgout = popen2('convert -quality %s -geometry %sx%s - -' % (quality, width, height), 'b') else: print "Started popen2\n" from popen2 import popen2 imgout, imgin = popen2('convert -quality %s -geometry %sx%s - -' % (quality, width, height)) print "Writing to stdin" imgin.write(origimg._IMdata()) print "Finished writing to stdin\n" imgin.close() newimg.write(imgout.read()) imgout.close() When I run zope in debug mode, I see "Started popen2" and the "Writing to stdin", but the process never gets to "Finished writing to stdin." Furthermore, zope will no longer respond to requests until I restart it. I already verified that origimg._IMdata() doesn't hang. I am using Photo 1.2.3 from http://zope.org/Members/rbickers/Photo/ with ImageMagick on Linux 2.2.20. Any hints? Thanks. Troy
I tried to reproduce the error on my w2k box (with Zope 2.7.0 on a regular installation of Python 2.3.3, not the bundled version.), but it didn't happen. Guess that reduces the problem to Linux. I sugest to place the offending line in a try statement, and see what happens: try: imgin.write(origimg._IMdata()) except: raise HTH Ausum ----- Original Message ----- From: "Troy Farrell" <troy@entheossoft.com> To: <zope@zope.org> Sent: Monday, February 16, 2004 10:53 AM Subject: [Zope] popen2 problem with Zope 2.7.0 and Photo
Greetings Zopistas. Since I've upgraded from 2.6.2 to 2.7.0, the following product code causes zope to hang:
elif engine == 'ImageMagick': # Use ImageMagick if sys.platform == 'win32': from win32pipe import popen2 imgin, imgout = popen2('convert -quality %s -geometry %sx%s - -' % (quality, width, height), 'b') else: print "Started popen2\n" from popen2 import popen2 imgout, imgin = popen2('convert -quality %s -geometry %sx%s - -' % (quality, width, height)) print "Writing to stdin" imgin.write(origimg._IMdata()) print "Finished writing to stdin\n" imgin.close() newimg.write(imgout.read()) imgout.close()
When I run zope in debug mode, I see "Started popen2" and the "Writing to stdin", but the process never gets to "Finished writing to stdin." Furthermore, zope will no longer respond to requests until I restart it. I already verified that origimg._IMdata() doesn't hang. I am using Photo 1.2.3 from http://zope.org/Members/rbickers/Photo/ with ImageMagick on Linux 2.2.20.
Any hints?
Thanks. Troy
_______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
On 02/16/2004 10:53 AM, Troy Farrell wrote:
Greetings Zopistas. Since I've upgraded from 2.6.2 to 2.7.0, the following product code causes zope to hang:
What exactly are you trying to do when it hangs? So far everything I've tried works. I'm running Fedora Core 1, Python 2.3.3, Zope 2.7.0, ImageMagick 5.5.6.
Any hints?
The "best" thing to do, IMO, would be to install PIL 1.1.3 or higher and use it instead of ImageMagick. As of 1.1.3, PIL has much better resize rendering which is as good as ImageMagick. Since PIL is a Python module, you won't have any piping issues. It's also faster at rendering. For it to render nicely, you'll need to change the following line (around 354) in Photo.py: from: img = img.resize((width, height)) to: img = img.resize((width, height), PIL.Image.ANTIALIAS) I have a little Python Script I used to convert my entire Photo collection from ImageMagick to PIL. I'll be happy to share if you want to go this route. -- Ron Bickers Logic Etc, Inc.
What exactly are you trying to do when it hangs? So far everything I've tried works. I'm running Fedora Core 1, Python 2.3.3, Zope 2.7.0, ImageMagick 5.5.6.
Interesting. I just accessing my 'view' page template. Eventually, the _resize method gets called and that's where the popen2 hangup happens. It's even more wierd that the same popen2 call doing the same thing works fine outside of Zope (i.e. interactive python prompt.) $ uname -a Linux bosco 2.4.20-gentoo-r5 #4 SMP Thu Oct 16 15:29:56 CDT 2003 i686 Intel(R) Xeon(TM) CPU 2.00GHz GenuineIntel GNU/Linux $ python Python 2.3.3 (#1, Feb 5 2004, 13:04:26) [GCC 3.2.3 20030422 (Gentoo Linux 1.4 3.2.3-r3, propolice)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
I'm wondering if my system is over optimized (-O3)...
Any hints?
The "best" thing to do, IMO, would be to install PIL 1.1.3 or higher and use it instead of ImageMagick. As of 1.1.3, PIL has much better resize rendering which is as good as ImageMagick. Since PIL is a Python module, you won't have any piping issues. It's also faster at rendering.
For it to render nicely, you'll need to change the following line (around 354) in Photo.py:
from: img = img.resize((width, height)) to: img = img.resize((width, height), PIL.Image.ANTIALIAS)
I have a little Python Script I used to convert my entire Photo collection from ImageMagick to PIL. I'll be happy to share if you want to go this route.
Please do share the script. I'm building PIL now :) Troy
On 02/16/2004 1:54 PM, Troy Farrell wrote:
I have a little Python Script I used to convert my entire Photo collection from ImageMagick to PIL. I'll be happy to share if you want to go this route.
Please do share the script. I'm building PIL now :)
Create a Python Script with the attached content in your root folder and call it through the Web. It will 1) change the default engine to PIL for all Photo Folders, 2) change the engine of all Photo objects to PIL, and 3) purge all the cached displays created by ImageMagick. It has been a while since I've used this. Let me know if you have any problems with it. -- Ron Bickers Logic Etc, Inc. """Regenerate ImageMagick images using PIL.""" print 'The following objects were upgraded:<br>\n' # Change default engine of all Photo Folders to PIL. folders = context.ZopeFind(context, obj_metatypes=['Photo Folder'], search_sub=1) for (id, folder) in folders: if folder.propertysheets.get('photoconf').getProperty('engine') == 'ImageMagick': folder.propertysheets.get('photoconf').manage_changeProperties(engine='PIL') print 'Photo Folder <tt>' + '/'.join(folder.getPhysicalPath()) + \ '</tt><br>\n' photos = context.ZopeFind(context, obj_metatypes=['Photo'], search_sub=1) # Change engine of all Photos to PIL and purge displays. for (id, photo) in photos: if photo.propertysheets.get('photoconf').getProperty('engine') == 'ImageMagick': print 'Purging %s<br>\n' % id photo.propertysheets.get('photoconf').manage_changeProperties(engine='PIL') photo.manage_purgeDisplays() print 'Photo <tt>' + '/'.join(photo.getPhysicalPath()) + \ '</tt><br>\n' return printed
Create a Python Script with the attached content in your root folder and call it through the Web. It will 1) change the default engine to PIL for all Photo Folders, 2) change the engine of all Photo objects to PIL, and 3) purge all the cached displays created by ImageMagick.
It has been a while since I've used this. Let me know if you have any problems with it.
Works perfectly and the thumbnails look as good as IM with the antialiasing. Thanks. Troy
Troy Farrell wrote at 2004-2-16 09:53 -0600:
Since I've upgraded from 2.6.2 to 2.7.0, the following product code causes zope to hang: ... imgout, imgin = popen2('convert -quality %s -geometry %sx%s - -' % (quality, width, height)) print "Writing to stdin" imgin.write(origimg._IMdata()) print "Finished writing to stdin\n" imgin.close() newimg.write(imgout.read()) imgout.close()
When I run zope in debug mode, I see "Started popen2" and the "Writing to stdin", but the process never gets to "Finished writing to stdin." Furthermore, zope will no longer respond to requests until I restart it.
This may be quite normal: As you might know: pipes have limited buffer only. When the buffers are full, you get a deadlock (as "write" waits for the buffer (to "convert") to get drained and this will only happen when the second buffer (from "convert") gets drained but you read this second buffer only after you have written everything). This is a well known problem with "popen2". You will need non-delaying IO to work (reliably) around it. -- Dieter
participants (5)
-
Ausum Studio -
Dieter Maurer -
Ron Bickers -
Ron Bickers -
Troy Farrell