You can put the code for the whole workflow inside Docker. However, in many cases, there are benefits for having multiple steps of execution as a workflow rather than wrapping everything into a single step as Docker. Workflow helps in conceptualizing the underneath operation, providing flexibility which step you want to operate, and resumability to start where it left off. For example, imaging that you have a variant calling pipeline. Breaking down analysis into QC, mapping, and actual variant calling is useful to train people and check where the error occurs. You would have flexibility to mix and match different tools for each step as well. Also, the output from each step could be used even if the downstream steps failed.
Of course, there are use cases where single step execution is preferable, especially in some production pipelines where all or none is a preferable way to track successful analysis.
On UKB-RAP, the most common way to construct workflows are the DNAnexus workflow GUI or write the WDL workflow. Other options such as building DNAnexus/CWL/Nextflow workflow from the script are also possible.
Comments
1 comment
You can put the code for the whole workflow inside Docker. However, in many cases, there are benefits for having multiple steps of execution as a workflow rather than wrapping everything into a single step as Docker. Workflow helps in conceptualizing the underneath operation, providing flexibility which step you want to operate, and resumability to start where it left off. For example, imaging that you have a variant calling pipeline. Breaking down analysis into QC, mapping, and actual variant calling is useful to train people and check where the error occurs. You would have flexibility to mix and match different tools for each step as well. Also, the output from each step could be used even if the downstream steps failed.
Of course, there are use cases where single step execution is preferable, especially in some production pipelines where all or none is a preferable way to track successful analysis.
On UKB-RAP, the most common way to construct workflows are the DNAnexus workflow GUI or write the WDL workflow. Other options such as building DNAnexus/CWL/Nextflow workflow from the script are also possible.
Please sign in to leave a comment.