We are downloading the raw accelerometry data (p90001) from physical activity measurements. However, the raw data in the ?activity? folder only showed 50000 items, and the other half items can not been seen or downloaded. How we can solve this?
For this, you might need to check with UKB team when the data would be available. I sent a request to DNAnexus data team, but they released all the data they received, so they would still need to ask for the data from UKB as well.
When you use the RAP to view bulk files, if there are more than 1000 files in a folder, only 1000 files will be shown in the list. This is only the most recently modified 1000 files. It is possible to apply a filter to the file search. For example, in Folder Activity>Epoch>10, start by clicking the ?Show/Hide Filters? icon. Select the ?Any Name? button, enter 101, click ?Apply?. This will show only those files in folder 10 with the string 101 in the name. There are fewer than 1000 of these, so all of them show in the list. Repeat this for strings 102 to 109, and you should be able to access all 2073 files in folder 10.
Once you can see all the files, they can be downloaded one at a time by selecting a single file, selecting the ?Download? button, selecting the download icon.
To download multiple files, select all files in the list (top little square next to the ?Name? column name), select the ?Download? button to generate download URLs. A wget command from a unix command line can then be used to download the files.
As an alternative to downloading the cwa files, you could consider using a JupyterLab session on the RAP to analyse the data in situ.
Comments
2 comments
For this, you might need to check with UKB team when the data would be available. I sent a request to DNAnexus data team, but they released all the data they received, so they would still need to ask for the data from UKB as well.
When you use the RAP to view bulk files, if there are more than 1000 files in a folder, only 1000 files will be shown in the list. This is only the most recently modified 1000 files. It is possible to apply a filter to the file search. For example, in Folder Activity>Epoch>10, start by clicking the ?Show/Hide Filters? icon. Select the ?Any Name? button, enter 101, click ?Apply?. This will show only those files in folder 10 with the string 101 in the name. There are fewer than 1000 of these, so all of them show in the list. Repeat this for strings 102 to 109, and you should be able to access all 2073 files in folder 10.
Once you can see all the files, they can be downloaded one at a time by selecting a single file, selecting the ?Download? button, selecting the download icon.
To download multiple files, select all files in the list (top little square next to the ?Name? column name), select the ?Download? button to generate download URLs. A wget command from a unix command line can then be used to download the files.
As an alternative to downloading the cwa files, you could consider using a JupyterLab session on the RAP to analyse the data in situ.
Please sign in to leave a comment.