[Zope] To thread or not to thread ...

[Michal Bencur]

Hi Terry,

I never done this before, but I think you should take care
about transaction commit which is done after each request.
So if you start a new thread, you probably have to get a new
connection to ZODB, do what you need to do, and commit it.


regards,
Michal

On Sat, Jun 22, 2002 at 06:11:43AM -0700, Terry Hancock wrote:
> Hi all,
> 
> As yet another iteration in a very image intensive product,
> I'm trying to build a tree of images. They are generated
> procedurally, and may differ from installation to installation
> (they don't currently, but they probably will).
> 
> So it's much more convenient to generate them procedurally
> in the manage_add<my-product>() function than to store them
> in, say, a .zexp file.  (I also tried making them on-the-
> fly, but that's a whole 'nother story).
> 
> Nevertheless, it takes quite awhile for PIL to chunk
> through all those images (currently 2880).
> 
> If I just use a straight iteration, the browser will
> time-out during the installation process, which is kind
> of ugly, and will no doubt freak out my potential user/
> administrator.  Also, it's not absolutely clear to me
> that Zope won't cut me off somehow if my request takes
> too long.  (Any reassurances/warnings on this point?).
> 
> Alternatively, I can put components of the installation
> process into separate threads.  This nicely drops me
> back into the management interface and runs the object-building
> process in the background.  However, I then have to think
> about checking it to make sure it's done (probably I can
> handle this, though I haven't yet -- I just check it
> manually later).
> 
> The big headache though, is that with the threaded version,
> I have folders disappear out of the management interface!
> 
> If I wait long enough, eventually they just start to
> disappear. Same thing if I restart. So the persistence
> machinery must not be working in the threaded case.
> 
> But why not?
> 
> The code is essentially the same, except that in the
> non-threaded version I just call the function, whereas
> in the threaded version, I create a thread to run it
> with threading.Thread().  I do this at the highest level
> in the loop, which means about 4 threads should be started,
> which I wouldn't think would tax my system (an earlier
> version with about 50 threads did cause a massive slowdown).
> 
> Both versions use ordinary 'Folder' objects and create
> them using the Zope API with manage_addFolder() calls
> on the parent folders.
> 
> Should I abandon threading here? (and just document
> this in the INSTALL file). Should I use full-fledged
> processes? Or am I just forgetting to do something?
> 
> (I've done almost no threaded programming before this --
> I did run a threaded test version of this code that
> generated the files on the filesystem outside of
> Zope. All my tests with that version went fine.).
> 
> Thanks in advance for any ideas,
> Terry
> 
> -- 
> ------------------------------------------------------
> Terry Hancock
> hancock@anansispaceworks.com       
> Anansi Spaceworks                 
> http://www.anansispaceworks.com 
> P.O. Box 60583                     
> Pasadena, CA 91116-6583
> ------------------------------------------------------
> 
> 
> _______________________________________________
> Zope maillist  -  Zope@zope.org
> http://lists.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists - 
>  http://lists.zope.org/mailman/listinfo/zope-announce
>  http://lists.zope.org/mailman/listinfo/zope-dev )