[Zope] dealing with scripts that take too long

[Andrew Langmead]

On Jan 25, 2006, at 5:17 PM, martin f krafft wrote:

>
> we're experiencing problems with certain maintenance scripts, which
> just take too long to complete, so that the browser resets the
> connection and Zope aborts the transaction.

If these are maintenance scripts that are kicked off manually by  
admins (as opposed to things that can be automated with cron) then  
maybe you can use a technique similar to one we use here.

For certain long running actions that our users need, we use the Zope  
Scheduler product. The clicks a button on a UI form, the form then  
adds the script to the schedule queue as a one shot action. The  
scheduler clock is actually running on an entirely separate Zope  
instance on a different machine, so that machines sole duty is to  
handle these async requests. The script, once completed updates its  
status, so revisiting the initial form will let the user know the  
status of their request. (we also give them a view into the schedule  
queue so if things are taking too long at least they know where they  
stand.)

The Zope server that processes the scheduler queue is set up  
differently, since it is never has to handle user input. It only has  
a single thread, and the cache size is larger than normal (the stock  
zope config is for four threads, and each thread has its own cache  
pool.)