... | @@ -38,3 +38,24 @@ Questions |
... | @@ -38,3 +38,24 @@ Questions |
|
---
|
|
---
|
|
1. Which parts of a task is declative (defined by attributes?) vs. procedural (Python?) ?
|
|
1. Which parts of a task is declative (defined by attributes?) vs. procedural (Python?) ?
|
|
2. Where does REMUS, Molequeue and Cumulus fit in ?
|
|
2. Where does REMUS, Molequeue and Cumulus fit in ?
|
|
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
# Comments 06-Jul-2016 (john)
|
|
|
|
### Context
|
|
|
|
I presume that this is a continuation of the previous workflow discussions, but to make sure I am calibrated:
|
|
|
|
* A "workflow" is analogous to a recipe for producing some desired engineering data.
|
|
|
|
* A "task" is analogous to one step in a workflow.
|
|
|
|
* And the "tasks" we are talking about are not automated or hands-off, but instead involve human-machine interaction, at least typically.
|
|
|
|
|
|
|
|
### Use Case Analysis
|
|
|
|
I think a set of use-case descriptions would be very helpful toward understanding the software requirements. To start, I can formalize the GSSHA simulation workflow that Amanda described to Bob and me last year (although I am not sure *when* I will do that...).
|
|
|
|
|
|
|
|
### Workflow datastore
|
|
|
|
The current description is focused on defining workflows and tasks. Looking ahead to the execution, the "workflow manager" will need some kind of repository to store/track the data involved in each individual workflow. Girder might be a good starting point for that infrastructure, but I don't want to get too far ahead of myself.
|
|
|
|
|
|
|
|
### External tasks
|
|
|
|
Although the main workflow focus is for things CMB does, we should design the workflow manager to support tools outside of CMB. As a simple example, I can foresee the need for a "review" task, in which one or more people "sign off" on some data/resource before it is considered ready for use in downstream tasks.
|
|
|
|
|
|
|
|
### Declarative vs. Procedural
|
|
|
|
I think this question is best addressed after one or more use-cases have been examined. But to start, I would vote for an approach like Ansible, which uses a declarative specification (yml) at the top level, that can invoke programs and procedural scripts when processed by the Ansible runtime. In our case, for example, a user could "initialize" a new workflow by specifying a yml file to the workflow manager. |