Deep-Learning on preprocessed MRIs on RAP

Esra Lenz

Hello,

I am kind of lost in the RAP since I am a Medical Doctor with some coding knowledge but by no means anyway familiar with working in the Cloud.

I am wondering if someone has a good setup/workflow for the following problem:

I want to use MRI images, for let's say predicting some outcomes.

My local code already uses Pytorch-Ligthning, Hyrda and Weights and Biases for logging.

 

What is the best way to do this? I don't see my code distributed in ipynb Notebooks and running there. My solution so far would be (also for debugging)

1. Preprocess or extract all the data (60.000 MRIs), by first loading them from BULK and then save them to my project. 

2. to make cloud-Instance with GPU, ssh-tunnel into it with VS-Code so I have a IDE, 

3. create a mount directory and mount my code into the instance (or just copy or clone it),

4.  install all the dependencies in the instance there via env file (probably with a lot of debugging of dependencies issues since the CUDA-Version is 11.4 of the GPUs), 

5. dx load all the needed Images from my Project in there 

6. run it and hope for the best

Does anybody have a better idea or solution for such a setup or goal? I would love to hear it. Do the Workflows help with that in anyway or is it a waste of time?

Best,

Esra

Comments

3 comments

Date Votes
  • Comment author
    Francesco La Rosa

    This is a great question, I would also be interested if someone is able to help.

    0
  • Comment author
    Oliver G The helpers that keep the community running smoothly. UKB Community team Data Analyst

    Hi Esra,

    Apologies for the slow reply. We are planning to release some resources on the UK Biobank GitHub (https://github.com/UK-Biobank) account and on this community site over the next few months to provide example code and describe methods. 

    In short, these will most likely involve creating docker files and images, and creating applets on DNAnexus. 

    The docker side of things should allow you to install and control your dependencies.

    The applets should allow you to input the relevant data, start a docker container, and perform the appropriate operations, and output the relevant data in parallel for each participant.

    Thanks,
    Ollie

    1
  • Comment author
    Oliver G The helpers that keep the community running smoothly. UKB Community team Data Analyst

    The following provides an example for using docker (although it uses the wdl workflow language rather than applets for the coordination of the jobs):
    https://github.com/UK-Biobank/UKB-RAP-Workflows/tree/main

    Here's some extra readings on applets on DNAnexus too:
    https://documentation.dnanexus.com/developer/apps/intro-to-building-apps

     

    0

Please sign in to leave a comment.