How do I call the table_exporter applet from my WDL?
I've tried running dxcomplier dxni as described here dxCompiler/ExpertOptions.md at develop · dnanexus/dxCompiler (github.com) but table-exporter isn't in my project (it's vended by DNAnexus) and I can't figure out how to point dxcompiler dxni at it properly.
As an alternative, I've tried just manually writing the WDL based on the instructions at that page:
task extraction_common {
input {
Array[String] field_names
String dataset = "/app46122_20220823045256.dataset"
String output_format = "TSV"
String coding_option = "RAW"
String header_style = "UKB-FORMAT"
}
output {
Array[File] csv = [""]
}
command {}
runtime {
dx_app: object {
type: "app"
id: "app-GKzZfz09ZgqVx5927xVfqBVp"
}
}
}
(having found the app ID from the URL App | Table exporter (dnanexus.com) ), and compiling the WDL works fine, but when I try to run a workflow using that task I get the error
dxpy.exceptions.InvalidInput: A reference to an execution given for "pcs___csv" referred to a field "csv" that does not exist in the output specification of the specified execution, code 422. Request Time=1674082583.584351, Request ID=1674082583824-840939
This is despite the fact that
dx describe table-exporter --json
includes
Output Spec csv (array:file)
Does anyone have a suggestion to make either of those two methods work?
----
Aside from the issue above, I worry that I'll run into the issue that the required input
Input Spec dataset_or_cohort_or_dashboard (record)
is supposed to be of type record and WDL doesn't have a record type. Will that be an issue? If so, will it be surmountable?
Comments
2 comments
For the bug, it might be easier for support team to help with this since they could access your project. You can submit the ticket through the platform or send request to ukbiobank-support@dnanexus.com.
To be honest, I never thought of compatibility between record type and WDL. It might be an issue as you said. I have never tried that.
If it's not require that you need to have your workflow in WDL. You could construct the workflow in UKB-RAP GUI as well. It's also not a loss of tracking between code and workflow. You can then use dx get <workflow_name> to get the dxworkflow.json that you could deploy it back using dx build within folder that contain dxworkflow.json.
If you want to go with WDL, using dx extract_dataset that Ondrej suggest might be a better solution. This way, you can have input as string. https://github.com/dnanexus/OpenBio/blob/master/dx-toolkit/dx_extract_dataset_bash.ipynb
Please sign in to leave a comment.