Former User of DNAx Community_47

Posts

Recent activity by Former User of DNAx Community_47

Sort by recent activity

Recent activity Votes

• How to tell that a job has failed due to an out of memory error when it completes with state "done"

  I am running an applet on the DNA Nexus RAP that pipes output from bcftools into another program. Sometimes the applet runs into insufficient memory... but it does not die with an error - it comple...

  • 18 December 2023 00:00
  • 2 followers
  • 3 comments
  • 0 votes

• DNA Nexus takes ages to run a job

  The past week or more, it takes many hours before any job that I submit starts running on DNA Nexus. This makes it nearly impossible to develop / debug an app or commands I want to run. Does anyone...

  • 02 December 2022 00:00
  • 2 followers
  • 2 comments
  • 0 votes

• How do you run a spark app?

  I have read the documentation on Spark apps (https://documentation.dnanexus.com/developer/apps/developing-spark-apps), but it's not clear to me how to actually run one.   For example, I have run th...

  • 17 November 2022 00:00
  • 3 followers
  • 7 comments
  • 0 votes

• Expected format for 500k WGS variants?

  I see that the public release of the 500k WGS data is expected in late 2023 (https://www.ukbiobank.ac.uk/enable-your-research/about-our-data/future-data-release-timelines).   Does anyone know in wh...

  • 10 November 2022 00:00
  • 2 followers
  • 5 comments
  • 0 votes

• How can you request an increased job limit?

  I have a lot of jobs to run to process WGS VCF data. I have heard that it's possible to request an increased quota for number of jobs running. I have emailed support to request this, but it has bee...

  • 10 November 2022 00:00
  • 1 follower
  • 1 comment
  • 0 votes

• Is it possible to run a high priority app (e.g. ttyd) while many jobs are queued?

  I would like a way to have interactive access on DNA Nexus, such as by running ttyd, while I also have many jobs (>100) currently running and more queued. I have tried to submit a new ttyd job, wit...

  • 10 November 2022 00:00
  • 3 followers
  • 4 comments
  • 0 votes

• How can I terminate all running/waiting jobs?

  I've submitted a few thousand jobs which I now realise may be using the wrong data. How can I terminate all of these as quickly as possible? Is there a better way than looping through job IDs calli...

  • 09 November 2022 00:00
  • 1 follower
  • 1 comment
  • 0 votes

• How can I get data from a DNA Nexus database into another format?

  I can create a DNA Nexus database, and confirm that it's present with dx describe. E.g.:   import dxpy db_name = "mydb" mt_name = "my_table" stmt = f"CREATE DATABASE IF NOT EXISTS {db_name} LOCATIO...

  • 07 October 2022 00:00
  • 3 followers
  • 4 comments
  • 0 votes

• Where does saved data go on a Jupyter spark cluster?

  I can process some test data with Hail on a Jupyter lab spark cluster, and for example export a filtered VCF. How can I get this data back up to my DNA Nexus project? Example:   import hail as hl b...

  • 07 October 2022 00:00
  • 1 follower
  • 1 comment
  • 0 votes

• The sample code to load a VCF file with Hail is failing for me when I try it on a single 6 Gb VCF from the whole genome pVCFs. Can you help me to resolve this error?

  Here is my code, and its output.I ran this by starting an analysis with a JupyterLab Spark Cluster with the default settings (mem1_hdd1_v2_x16 with 2 workers).  from pyspark.sql import SparkSession...

  • 30 September 2022 00:00
  • 2 followers
  • 3 comments
  • 0 votes