UKB RAP virtual machines with GPU
How can we have virtual machines with GPU cards to allow us run our deep learning models?
Since the DNAnexus builds on top of AMS anyway, can we access the EC2 instances like environment from the DNAnexus but with all the data from our projects available?
Comments
9 comments
I second this question. I accessed some GPU instances in the current rate card using ttyd; it seems that Nvidia V100 with 16GB memory is the highest available configuration.
Any information on the availability of higher memory AWS GPU instances over RAP is appreciated.
For our run, we typically need to train the models several GPUs for a few weeks. TTYD is only for ad-hoc command line jobs. What should we do if we want to do long-term training and be able to monitor the training process?
Yes sorry I didn't mean to answer your question but to reiterate that we have the same question. As far as I know, you may need to use a JupyterLab instance with GPU support (ML feature) or code your analysis within a Docker image and compile to an app.
We're particularly interested in knowing which instances offer GPUs with the highest memory for processing neuroimaging and genomic data. Hopefully, someone from the community can provide a clearer answer.
Hey Sourena!
Did you find a solution and answer to your question?
What would be the best workflow for such analyses? (Is it really the App and do I need to use WDL then?)
I am facing the exact same problems. I want to do Deep-Learning on preprocessed MRI-Images.
The only way I see this currently happening is using utilizing Nividia’s toolkit container to setup your docker and running it though Swiss Army Knife. I could be completely wrong as I’m actually busy working on this exact problem since the beginning of this week. I will probably put the solution on git if anyone is interested.
Donphi please do. I have no idea how Swiss Army Knife works. I am sure many people are in similar shoes :D
Will do. I’ll keep you posted.
Hi Donphi Hang Yuan have you found more information on GPU on RAP (i.e. vram, number of GPUs per node)? I'm trying to do a deep learning project on MRI images on RAP and would like to see how feasible it is. I heard that insufficient number of GPUs can also be a reason to apply for RAP exemption. Has any of you applied to that?
Trang Minh Nguyen as my current understanding goes, if your ML project only require 1-2 GPU cards or can be run on a notebook then just use the RAP notebook as it is.
For anything else, I am not aware of any scalable solution in its current form. So an exemption application might be the only way out.
Please sign in to leave a comment.