It's easy enough in the user interface to manually select the input files I need. But when it comes to a bash script or the interactive CLI, it seems like I have to individually list every single input file. Using an asterisk like "*.cram" works for the cmd line, but not for the input files I wish to dx download/mount.
Any easy way to do it, considering I need to mount over a thousand of files in one job?
Thanks,
Zeid
Once you have the list with desired files, you could iterate those with loop and download / mount / process?
2) What if you "generate"/"construct" your full submission command? See pseudocode:
for i in (dx ls / dx find data):
command_builder += " --in $i"
> to_be_inserted
dx run app-myapp {$to_be_inserted}
3) What if you input txt containing a list of files? Inside the script, you could then iterate through this list and do dx download, process data, remove data and continue with next file.
Comments
1 comment
1) On the worker, what if you list the files via "dx find data"?
https://documentation.dnanexus.com/user/objects/searching-data-objects#searching-objects-with-other-criteria
Once you have the list with desired files, you could iterate those with loop and download / mount / process?
2) What if you "generate"/"construct" your full submission command? See pseudocode:
for i in (dx ls / dx find data):
command_builder += " --in $i"
> to_be_inserted
dx run app-myapp {$to_be_inserted}
3) What if you input txt containing a list of files? Inside the script, you could then iterate through this list and do dx download, process data, remove data and continue with next file.
Please sign in to leave a comment.