[Zope-CMF] Project-style workflows, DCWorkflow, SQL, etc.

Phillip J. Eby pje@telecommunity.com
Tue, 02 Oct 2001 13:24:57 -0500


At 11:56 PM 10/1/01 +0000, Kent Polk wrote:
>On 1 Oct 2001 15:05:01 -0500, Phillip J. Eby wrote:
> > In this model, a "request" is just another waiter/waitee, like a 
> task.  The
> > main difference is that a request has a generic workflow - that of
> > responding to a request to do something.  The receiver of a request 
> (who is
> > what the request is "waiting for") then creates dependencies from the
> > request to the tasks that are needed to fulfill the request.  In a more
> > advanced system, specialized request types could make creation of the
> > standard tasks automatic - although in that case it may make sense to
> > simply customize the workflow and make it purely a task rather than a
> > request.  In general, it has seemed to us, however, that there is value in
> > distinguishing the tracking of a request to do something, from the actual
> > doing of it.
>
>Herein lie my conceptual problems with trying to use a CMF item to
>broker workflow actions on the behalf of its 'subworkflows', etc.
>I'm convinced that it shouldn't be managed by an item, but by a
>workflow management task of some sort, but how would you identify/manage
>a wait task in the CMF?

I don't know what you mean by "managed by an item".  But to try and explain 
better what a CMF implementation of this would look like, consider that CMF 
allows different workflow definitions to exist for the same base object 
implementation.  So if you have, for example, different processes which 
must be followed for "sample sheets", then you can define different 
workflows for them.  The base "sample sheet" object remains the same, but 
the workflow can be different.

If you derive all your content objects from a class that can handle links 
to/from other objects, then those workflows which require subworkflows 
simply create content objects with the correct workflow selected, and link 
them as dependencies.

I have little experience with actual CMF implementations at this point, so 
details of *how* you do that is best left to the CMF gurus.  In theory, it 
should be possible to do everything you want with little extension to the 
CMF.  If you were doing a ZODB-based implementation, you could probably 
even use ObjectManager as a base and use containment to manage 
dependencies; using some kind of "reference" object if you needed to have 
more than one item waiting for the same sub-item.  A similar "reference" 
object could be used in order to post a task to a worklist.



> > Anyway, this model is very simple and requires only two basic interfaces;
> > the vast majority of objects in the system will implement both.  People,
> > tasks, requests, etc. are all both waiters/waitees, requesters/responders,
> > dependents/dependencies, or whatever you want to call them.  Most of the
> > objects can be considered state-machines which go through a set of
> > transitions ala DCWorkflow.  Events on waited-for objects can trigger
> > transitions in their waiters, which may use guard conditions that check
> > whether there are any objects still being waited-for.
>
>I think it would be important for the current CMF items to
>transparently participate in this sort of workflow when they have
>been created by the workflow. Basically, their own workflow interacts
>with the project workflow task, as well as the rest of the CMF.
>
>Somehow...

Well, as long as you're defining all the workflows, this is 
straightforward.  You simply have the child workflows send a standard set 
of messages to their waiters, e.g. "waiter.subtaskDone(self)", 
"waiter.subtaskAborted(self)", etc.  If these messages are generic, then 
workflows which send them can be "plugged into" any standard parent 
workflow.  If you have more specific messages, then it can only be plugged 
into parent workflows which support those specialized messages.

Thing is, it's all up to you how you want to do this.  You define your own 
API.  One simple way to do it, though, would be to use the CMF's "workflow 
action" framework to message directly from the child workflow to the parent 
workflow - which would make the content object types for both parent and 
child irrelevant!



> > Although this model is simple, implementations can be arbitrarily
> > complex.  For example, you may want people or "worker" objects to be able
> > to do things like send e-mail notifications (or page/instant-message them)
> > when something is added to the person object's list of objects "waiting
> > for" them.
> >
> > DCWorkflow is a bit primitive in some of these areas, but there's nothing
> > in the above model that's inherently beyond its capabilities.  Your
> > principal difficulties would be in making it clear how these things
> > interact with one another, and perhaps creating some libraries or base
> > classes for dependency management, so that the guards, etc. in your
> > DCWorkflows are not too complicated to write.
>
>The complexity at this point can be a serious drawback. The system
>needs to be fairly easily comprehendable, and the options I see
>for a complex situation worry me.


You know, if there are enough people with both an interest in these types 
of workflows and good knowledge of the CMF workflow framework, perhaps a 
Wiki to define a workflow-to-workflow interaction standard might be in 
order.  That would probably fix most of the complexity issue.  Perhaps the 
CMF workflow API could be extended to allow for it explicitly, which might 
be even better.  (Assuming there isn't already an easy idiom that can be 
specified using the existing API.)



> > tends to be a fairly infrequent action where one needs to create a
> > dependency to an already existing task, so such use cases can usually be
> > addressed on a case by case basis.  For example, in the case of an ISP,
>
>We have several feedback loops where data is determined to be of
>insufficient quality, etc. and it must be reexamined. Some of these
>should be arranged for by a workflow filter, but some will be done
>by humans. Will have to take a closer look at that.

Perhaps what I was saying was unclear.  I don't see how what you're saying 
is relevant to the above.  Feedback loops are part of a single workflow, 
and I was speaking about the need to create a dependency between two 
pre-existing tasks.  For example, if two projects were waiting for the 
analysis of a single DNA sample to be completed.  While the first project 
might create the "DNA analysis" task, the second would have to find it and 
link to it, rather than creating a new task.  Make sense?



> > they may receive many requests from customers regarding a network outage,
> > and so they might need to make all these requests "wait for" a single NOC
> > ticket, in which case their requests need to have the ability to find and
> > link to NOC tickets.  But then you know in your specific design where to
> > look for NOC tickets.  Make sense?
>
>I think so. Most of our workflow is composed of serial and parallel
>actions that are pretty fixed, but we do have to determine a way
>to handle transients.

Huh?  See above.  I was still talking about an instance where two projects 
depend on the same task being completed.  In the example, multiple 
customers' "my site is down" tickets wait for one "fix the network" ticket 
before they can be resolved.



> > Thus, at a framework level, it suffices to have a registry (such as the
> > existing Zope/CMF type registries) for selecting types and creating new
> > waiters, and a way to look up users or worklists (such as some 
> extension to
> > the Membership machinery), and perhaps a standard way to find
> > "Organization" or "Team" objects (for department, role, or skill-based
> > worklists, as opposed to specific individuals).
>
>This issue of group or team activities keeps coming up. I wish we
>could find a solution to it, and not one that requires that CMF
>items be structured in a storage heirarchy to accomplish it.

It doesn't.  The standard DCWorkflow "worklist" idiom works just fine for 
this.  To do security correctly without a group-structured hierarchy, 
however, you might need to use local roles on the objects.



> > Regardless of your workflow implementation or even specific DB software, I
> > do think that the WarpCORE design patterns will make your SQL design
> > easier; specifically, it's geared to polymorphism, hierarchies, and
> > references to external objects (e.g. data stored in those "evil scientific
> > software" apps you mentioned previously).
>
>How is WarpCore doing? Been too busy to keep up with it.

I've been too busy to work on TransWarp much lately.  WarpCORE has long ago 
been in production use as a base schema for our Sybase application 
databases, however, and Ty has made prototype versions of the WarpCORE 
kernel schema for Firebird/Interbase and PostgreSQL.  I've also begun a 
draft of documentation for the WarpCORE design patterns and kernel, but 
have no schedule for completing it.