Hi all,
this is my my first post here and as a newbie I'm afraid I'll have a bunch of questions.
So, here is my first one. I saw that there is a support for nextflow ( https://documentation.dnanexus.com/user/running-apps-and-workflows/running-nextflow-pipelines ) but the interface on UKBB doesn't show the menu titled "import workflow" (it stops at "Import AWS S3") . Is it something related to my authorizations or the feature is not implemented for UKBB ?
is there any simple example for NF specific to dnanexus available somewhere ? (for example, what's in `executor` ?...)
thanks,
Pierre
In general, UKB-RAP has everything that DNAnexus platform has and even more features like Rstudio. However, the advance support for Nextflow is currently only for DNAnexus platform users with licensing.
What is your usual bioinformatics workflow you used in your development? Besides basic Nextflow support, you could also build workflow in UI or using WDL and CWL as well. Or would you think that the Advance support of Nextflow would be important for UKB-RAP community?
Currently, the advanced support is only available for people who have Apollo/Titan and use it on DNAnexus. If you works for the company/institution that use DNAnexus, could you E-mail us the info at community@dnanexus.com? We can try to check if those institution has Titan/Apollo and whether it is possible to grant you the access to our Advance Nextflow support.
@Pierre LIndenbaum? I discussed with product team and the advance nextflow support would require more than just flip the permission flag. There will need to be discussion at strategic level and also some engineering work. For now, I would recommend using Cloud workstation for Nextflow which would be quite limit in term of utility or use WDL instead.
I and people in my lab would also like to use a fully featured nextflow on the RAP. Our entire analysis pipeline was developed in Nextflow because it could run across multiple HPCs before the RAP was enforced. Now we can't use it on UK Biobank data. Why is it not activated on RAP? Is there any plan to make it available?
Comments
8 comments
There are two levels of Nextflow support on DNAnexus.
1 Basic support that wraps everything in an applet https://ukbiobank.dnanexus.com/app/nextflow_pipeline_importer
2 Advance support that supports Nextflow workflow https://documentation.dnanexus.com/user/running-apps-and-workflows/running-nextflow-pipelines
In general, UKB-RAP has everything that DNAnexus platform has and even more features like Rstudio. However, the advance support for Nextflow is currently only for DNAnexus platform users with licensing.
What is your usual bioinformatics workflow you used in your development? Besides basic Nextflow support, you could also build workflow in UI or using WDL and CWL as well. Or would you think that the Advance support of Nextflow would be important for UKB-RAP community?
thanks !
Sorry, I made a mistake with my latest answer. You are the first RAP user that wants to use Nextflow, and we have some miscommunication internally.
Let me try again:
There are two levels of Nextflow support on DNAnexus.
1 Install and run it in Cloud workstation. My colleague will share an example on this soon after testing.
2 Advance support that supports Nextflow workflow
https://ukbiobank.dnanexus.com/app/nextflow_pipeline_importer
https://documentation.dnanexus.com/user/running-apps-and-workflows/running-nextflow-pipelines
Currently, the advanced support is only available for people who have Apollo/Titan and use it on DNAnexus. If you works for the company/institution that use DNAnexus, could you E-mail us the info at community@dnanexus.com? We can try to check if those institution has Titan/Apollo and whether it is possible to grant you the access to our Advance Nextflow support.
Here are steps for getting started using nextflow on the cloud workstation
@Pierre LIndenbaum? I discussed with product team and the advance nextflow support would require more than just flip the permission flag. There will need to be discussion at strategic level and also some engineering work. For now, I would recommend using Cloud workstation for Nextflow which would be quite limit in term of utility or use WDL instead.
@Alexandra Lee? @Chai Fungtammasan? thank you both for your answers. I'm still looking for the best solution. Although I'm a nextflow user, I was able to run a simple pipeline using WDL ( but I still have some problems ... https://community.dnanexus.com/s/question/0D5t000004RYhxFCAT/wdl-specify-directory-as-input ), so I might focus on WDL.
Hi,
I and people in my lab would also like to use a fully featured nextflow on the RAP. Our entire analysis pipeline was developed in Nextflow because it could run across multiple HPCs before the RAP was enforced. Now we can't use it on UK Biobank data. Why is it not activated on RAP? Is there any plan to make it available?
Seconding Olivier's comment – we already have our own nextflow workflows in-house.
Please sign in to leave a comment.