Moving preprocessed bulk images to the RAP

James C Porter

I have already preprocessed bulk downloaded images as per the recent changes I will need to work using the RAP once I am at the stage where I need to use updated data. I have looked through the FAQ and can't find any guidance on how workflows with bulk images works. 

Images on the RAP:

  • Is there limits to the amount that can be uploaded to the RAP? I have over 560gb of processed images (so far)
  • How does storing pre-processed bulk data images work on the RAP? Surely I am not going to have to preprocess them and then pay constantly to store this data? Or will I need to run days of preprocessing everytime I want to do anything with the images?
  • If there is a way to approach this without incurring constant storage costs for the preprocessed images, how can I make use of my existing preprocessed images?

 

General:

  • I am a 1st year PHD student who has invested most of my available budget into a high-end machine to store and process these images. Is my machine now going to be a waste once I use updated UKB data? I am aware there are early-career researcher credits but they don't bring back my lost time and my actual research budget.
  • As far as I can tell, the RAP doesn't have excel? What about an IDE like pycharm?

 

Comments

1 comment

Date Votes
  • Comment author
    Daisy V The helpers that keep the community running smoothly. UKB Community team Data Analyst

    Hi James, 

    There are no limits to the amount of data that you can upload to the UKB-RAP. 

    Please note that whilst data dispensed into your project by the UKB-RAP does not incur storage charges, you will be charged for any data that you generate. You can see how much this will cost here.  

    You can continue to use your local machine to work on images that you have already downloaded - please see here for more detail. 

    Integrated Development Environments (IDEs), including JupyterLab and RStudio are available as development environments. You can find out more about how to use JupyterLab and RStudio on the UKB-RAP here in our Github notebooks. DNAnexus also provide extra information about how to bring your workflow to the platform in their Gitbook here. 

    Excel is not currently available on the UKB-RAP. We are always keen to hear from users and the feedback will guide the product development roadmap. 

    Thank you for getting in touch, we hope this has been helpful. 

    0

Please sign in to leave a comment.