Accessing/interacting with T1 brain MRI NIFTI/DICOM files in UKB RAP for processing/analysis
Hello, I am new to using UKB RAP and am unfamiliar with the toolboxes/resources that I can use to access/interact with the bulk data. I understand that currently brain MRI raw images are stored in a .zip file in the UKB RAP. As I am keen on using the raw data for processing and segmentation work, may I know how I can use them within the platform (Since download of bulk files are no longer an option moving forward from the policy change)? These are some of the things I have tried:
1. Using the pipeline from “https://github.com/dnanexus/OpenBio/blob/master/image_processing/image_processing_FSL.ipynb” run on the UKB RAP Jupyter notebook environment to access the zip file of one individual participant (for testing purposes). This failed as the directory which I used to lead to the zip file ended up being a HTML file instead of the full .zip file
2. Uploading the compressed folder for the pipeline “https://git.fmrib.ox.ac.uk/falmagro/UK_biobank_pipeline_v_1” in a folder created in our UKB RAP project and attempting to extract the file to use the pipeline. This failed as well as the folder is in a Read-only format
Thank you for your help and I look forward to any tips and advice that I can use to start my analysis. Thank you.
Comments
2 comments
Also, if I may ask an additional question here. Is the video tutorial titled "Image Analysis on the UK Biobank Research Analysis Platform" (https://www.youtube.com/watch?v=d6uEl4U8NEM) still applicable with the new policy that was implemented? The reason I am asking this is because there are still parts of the tutorial that mentions using the “dx download” command to download files, processing them and subsequently uploading the processed data back.
If this is not very accurate, could you help to guide me to the more updated tutorial for accessing the raw brain MRI files? Thank you
Hello James,
I was also very confused at the beginning. I think this tutorial may help you better, even though the example is not on imaging processing (https://www.youtube.com/watch?v=LC3JcBYj-Mo). If you look at the development cycle at 8:12, you see the schematic of how you can test your pipeline.
Basically, on your local computer, you can try building a web app using dx-app-wizard (install dx-toolkit, check out https://dnanexus.gitbook.io/uk-biobank-rap/working-on-the-research-analysis-platform/running-analysis-jobs/custom-app and https://documentation.dnanexus.com/developer/apps/intro-to-building-apps). I had a simple bash script that use a docker image with FSL etc, which had been saved as .tar.gz and uploaded to my project page using dx upload. You then compile the app with dx build, which upload your app to your project.
The first thing your main bash script (or any other language) inside your web app should do is to grab the input file (in the case of imaging bulk file, individual bulk zip folder) and download it to the cloud computing workstation using dx download. So here, dx download is being used to transfer the input zip file to the worker computers, and not to your own local computer (trying to do so would result in error, as it is no longer available for download). Then, it should be followed by unzip command to unzip the entire folder or selectively if you know which files you need from the zipped bulk file. Once this is done, you can follow it up with any manipulation/call to other scripts and at the end, return the output file you define in the app wizard.
Running your app via dx run MyApp will execute your script, and when it's finished and it finds the specified output, it returns this file to your project page.
I think your first approach should work if done correctly, but I didn't try it myself so I'm not sure what you did wrong. In any case, if you need to batch process the images with your pipeline, I'm not sure using the jupyter notebook instance is so useful.
Once my batch jobs are running correctly, I will try to post it publicly on github so it can help others in a similar situation.
Please sign in to leave a comment.