More info: I've probably missed something in the docs somewhere (sorry!), but what is the recommended way to use git with the UKB RAP?
Ideally I would like to have a repo with my analysis code in hosted on GitHub, and push and pull to that from my project on the UKB RAP (obviously not committing any data!). Is that possible, and if so, how? Or do I need to work around it in another way?
I can share some approach, and I will let our expert chime in if they have a better method.
I would usually bring up interactive workstation like ttyd or cloud workstation. Then if I run git clone using https method rather than SSH method, I was able to clone public git repo to those interactive workstation. After that, I would upload the script I want to the project using dx upload.
For example git clone https://github.com/Arkarachai/STR-FM.git
Then I get in those folder and run dx upload the file I need. You can use dx upload -r to recursively upload files under the path if there are multiple files.
Note that this is working only for pull. For pushing, you would probably need to log into github account from interactive workstation.
I think there might be a better trick if we use workstation that has dxfuse. I will let our expert confirm that.
As it was mentioned in one of the previous Community posts - https://community.dnanexus.com/s/question/0D5t000003yKtvaCAC/best-practices-for-backups-on-rap - I would just push major code updates to your git repo to backup your code and version. DNAnexus project is a permanent storage and the notebooks are created with versions. I believe that official integration with git does not exist.
In addition to what Chai said, I would use e.g. Cloud workstation or JupyterLab to edit my code (not my daily practice) and since these two tools have Terminals, your favourite git service should be available there or you can install git client there.
Also as a UKB RAP developer, I am not aware of any "dx git" command that would be integrated to permanent storage to enable working with git directly. One option would be to reclone the repository every session (or not exactly every session, but those sessions when you do a significant modification so you can create a new git branch etc.).
Typically, I work with git as it is described below:
I download/upload my notebooks between my local laptop with git project cloned (just code, not data), permanent storage (UKB RAP project) and temporary cloud worker (JupyterLab). In case I request pushing a new code version to the git - my notebook needs to be pushed to git, I download it from DNAnexus either to my local or cloud worker and push it to git repo.
Comments
4 comments
More info: I've probably missed something in the docs somewhere (sorry!), but what is the recommended way to use git with the UKB RAP?
Ideally I would like to have a repo with my analysis code in hosted on GitHub, and push and pull to that from my project on the UKB RAP (obviously not committing any data!). Is that possible, and if so, how? Or do I need to work around it in another way?
I can share some approach, and I will let our expert chime in if they have a better method.
I would usually bring up interactive workstation like ttyd or cloud workstation. Then if I run git clone using https method rather than SSH method, I was able to clone public git repo to those interactive workstation. After that, I would upload the script I want to the project using dx upload.
For example git clone https://github.com/Arkarachai/STR-FM.git
Then I get in those folder and run dx upload the file I need. You can use dx upload -r to recursively upload files under the path if there are multiple files.
Note that this is working only for pull. For pushing, you would probably need to log into github account from interactive workstation.
I think there might be a better trick if we use workstation that has dxfuse. I will let our expert confirm that.
As it was mentioned in one of the previous Community posts - https://community.dnanexus.com/s/question/0D5t000003yKtvaCAC/best-practices-for-backups-on-rap - I would just push major code updates to your git repo to backup your code and version. DNAnexus project is a permanent storage and the notebooks are created with versions. I believe that official integration with git does not exist.
In addition to what Chai said, I would use e.g. Cloud workstation or JupyterLab to edit my code (not my daily practice) and since these two tools have Terminals, your favourite git service should be available there or you can install git client there.
Also as a UKB RAP developer, I am not aware of any "dx git" command that would be integrated to permanent storage to enable working with git directly. One option would be to reclone the repository every session (or not exactly every session, but those sessions when you do a significant modification so you can create a new git branch etc.).
Typically, I work with git as it is described below:
I download/upload my notebooks between my local laptop with git project cloned (just code, not data), permanent storage (UKB RAP project) and temporary cloud worker (JupyterLab). In case I request pushing a new code version to the git - my notebook needs to be pushed to git, I download it from DNAnexus either to my local or cloud worker and push it to git repo.
Please sign in to leave a comment.