Creating large datasets

Karine Alcala

Hi,

 

i'm trying to use the UKB platform since this morning. Following this tutorial (https://github.com/dnanexus/OpenBio/blob/master/dx-toolkit/dx_extract_dataset_r.ipynb) I successfully found my data. When I'm doing a small selection of variables, I can go through the process and save it as a csv.

But when I have more variables, I have the following error: 

For this one, I tried to get all the following fields, for all participants: "40005","40006", "40011","40000", "40012","40007" (cancer and death outcomes)

When I'm selecting only 40005, it's working, so my script is fine.

What can I do to retrieve these data? Should I do it in little pieces only?

Thanks

Karine

 

 

Comments

2 comments

Date Votes
  • Comment author
    Daisy V The helpers that keep the community running smoothly. UKB Community team Data Analyst
    • Edited

    Hi Karine,

    dx extract_dataset has a limit to the number of columns it can extract, and I suspect this is what is happening here. To extract a larger number of fields to a csv, you can use the Table Exporter app. 

    There is a notebook here giving an example of generating a field list and running Table Exporter. You can also run the app from the GUI - please note that you must specify the entity (e.g participant for the fields you have specified above).

    Hope this helps!

    0
  • Comment author
    Karine Alcala

    Thanks so much, I will try that today!

    0

Please sign in to leave a comment.