Job priority trade-off?

Dr. Mc. Ninja

Hi,

If I use normal priority, I initially get a spot instance, which is cheaper, but can be ‘killed’ without warning (clear enough).

I *think* that if a job waits 15 mins for a spot instance, it then gets an ‘on-demand’ instance, which costs more, but won't be killed.

If I use high priority, I get on-demand out of the gate.

I *believe* that on-demand jobs are about 5x the cost of spot?

If I use low priority I *think* I'll never get on-demand, waiting as long as it takes to get spot?

1) I can't find a link to cost

2) Is it fair to say that if my job tries to finish on spot fewer than 5 times (guess) that it will be cheaper than on-demand?

Is the above fair?

 

Many thanks 

Comments

4 comments

Date Votes
  • Comment author
    Rachael W The helpers that keep the community running smoothly. UKB Community team Data Analyst

    Hi Dr,

    the 2026 rate card can be found here: https://20779781.fs1.hubspotusercontent-na1.net/hubfs/20779781/Product%20Team%20Folder/Rate%20Cards/BiobankResearchAnalysisPlatform_Rate%20Card_Current.pdf  and includes the different rates for spot and for on-demand.

    (It looks more like 2 or 3 times rather than 5).

     

    1
  • Comment author
    Dr. Mc. Ninja

    Thanks so much. Here is a visualisation that's useful for me:

     

    0
  • Comment author
    Rachael W The helpers that keep the community running smoothly. UKB Community team Data Analyst

    Thanks for sharing that.

    (I hadn't realised how much the saving rate varied)

    1
  • Comment author
    Dr. Mc. Ninja

    No problem, happy to share the full sheet if it's useful (I guess I may as well):

     

    rap_instance_type,aws_instance_type,version,storage_class,storage_rank,gpus,memory_class,vcpus,memory_gib,storage_gib,storage_type,on_demand_gbp_per_hr,spot_gbp_per_hr,spot_saving_rate
mem1_ssd1_v2_x2,c5d.large_v2,2,1,1,0,1,2,4,50,SSD,0.0628,0.0168,3.74
mem1_ssd1_v2_x4,c5d.xlarge_v2,2,1,1,0,1,4,8,100,SSD,0.1256,0.0336,3.74
mem1_ssd1_v2_x8,c5d.2xlarge_v2,2,1,1,0,1,8,16,200,SSD,0.2512,0.0672,3.74
mem1_ssd1_v2_x16,c5d.4xlarge_v2,2,1,1,0,1,16,32,400,SSD,0.5024,0.1344,3.74
mem1_ssd1_v2_x36,c5d.9xlarge_v2,2,1,1,0,1,36,72,900,SSD,1.1304,0.3024,3.74
mem1_ssd1_v2_x72,c5d.18xlarge_v2,2,1,1,0,1,72,144,1800,SSD,2.2608,0.6048,3.74
mem2_ssd1_v2_x2,m5d.large_v2,2,1,1,0,2,2,8,75,SSD,0.0716,0.0168,4.26
mem2_ssd1_v2_x4,m5d.xlarge_v2,2,1,1,0,2,4,16,150,SSD,0.1432,0.0336,4.26
mem2_ssd1_v2_x8,m5d.2xlarge_v2,2,1,1,0,2,8,32,300,SSD,0.2864,0.0672,4.26
mem2_ssd1_v2_x16,m5d.4xlarge_v2,2,1,1,0,2,16,64,600,SSD,0.5728,0.1344,4.26
mem2_ssd1_v2_x32,m5d.8xlarge_v2,2,1,1,0,2,32,128,1200,SSD,1.1456,0.2688,4.26
mem2_ssd1_v2_x48,m5d.12xlarge_v2,2,1,1,0,2,48,192,1800,SSD,1.7184,0.4032,4.26
mem2_ssd1_v2_x64,m5d.16xlarge_v2,2,1,1,0,2,64,256,2400,SSD,2.2912,0.5376,4.26
mem2_ssd1_v2_x96,m5d.24xlarge_v2,2,1,1,0,2,96,384,3600,SSD,3.4368,0.8064,4.26
mem3_ssd1_v2_x2,r5d.large_v2,2,1,1,0,3,2,16,75,SSD,0.0922,0.0168,5.49
mem3_ssd1_v2_x4,r5d.xlarge_v2,2,1,1,0,3,4,32,150,SSD,0.1844,0.0336,5.49
mem3_ssd1_v2_x8,r5d.2xlarge_v2,2,1,1,0,3,8,64,300,SSD,0.3688,0.0672,5.49
mem3_ssd1_v2_x16,r5d.4xlarge_v2,2,1,1,0,3,16,128,600,SSD,0.7376,0.1344,5.49
mem3_ssd1_v2_x32,r5d.8xlarge_v2,2,1,1,0,3,32,256,1200,SSD,1.4752,0.2688,5.49
mem3_ssd1_v2_x48,r5d.12xlarge_v2,2,1,1,0,3,48,384,1800,SSD,2.2128,0.4032,5.49
mem3_ssd1_v2_x64,r5d.16xlarge_v2,2,1,1,0,3,64,512,2400,SSD,2.9504,0.5376,5.49
mem3_ssd1_v2_x96,r5d.24xlarge_v2,2,1,1,0,3,96,768,3600,SSD,4.4256,0.8064,5.49
mem1_hdd1_v2_x2,c5.large_v2_hdd,2,1,4,0,1,2,4,200,EBS-st1,0.0796,0.0314,2.54
mem1_hdd1_v2_x4,c5.xlarge_v2_hdd,2,1,4,0,1,4,8,400,EBS-st1,0.1592,0.0628,2.54
mem1_hdd1_v2_x8,c5.2xlarge_v2_hdd,2,1,4,0,1,8,16,800,EBS-st1,0.3184,0.1256,2.54
mem1_hdd1_v2_x16,c5.4xlarge_v2_hdd,2,1,4,0,1,16,32,1600,EBS-st1,0.6368,0.2512,2.54
mem1_hdd1_v2_x36,c5.9xlarge_v2_hdd,2,1,4,0,1,36,72,3600,EBS-st1,1.4328,0.5652,2.54
mem1_hdd1_v2_x72,c5.18xlarge_v2_hdd,2,1,4,0,1,72,144,7200,EBS-st1,2.8656,1.1304,2.54
mem1_hdd1_v2_x96,c5.24xlarge_v2_hdd,2,1,4,0,1,96,192,9600,EBS-st1,3.8208,1.5072,2.54
mem3_ssd2_v2_x2,i3.large,2,2,1,0,3,2,15.25,475,SSD,0.0988,0.0988,1.00
mem3_ssd2_v2_x4,i3.xlarge,2,2,1,0,3,4,30.5,950,SSD,0.1976,0.1976,1.00
mem3_ssd2_v2_x8,i3.2xlarge,2,2,1,0,3,8,61,1900,SSD,0.3952,0.3952,1.00
mem3_ssd2_v2_x16,i3.4xlarge,2,2,1,0,3,16,122,3800,SSD,0.7904,0.7904,1.00
mem3_ssd2_v2_x32,i3.8xlarge,2,2,1,0,3,32,244,7600,SSD,1.5808,1.5808,1.00
mem3_ssd2_v2_x64,i3.16xlarge,2,2,1,0,3,64,488,15200,SSD,3.1616,3.1616,1.00
mem2_ssd2_gpu1_v2_x4,g6.xlarge_gpu1_v2,2,2,1,1,2,4,16,250,SSD,0.5576,0.512,1.09
mem2_ssd2_gpu1_v2_x8,g6.2xlarge_gpu1_v2,2,2,1,1,2,8,32,450,SSD,0.6776,0.608,1.11
mem2_ssd2_gpu1_v2_x16,g6.4xlarge_gpu1_v2,2,2,1,1,2,16,64,600,SSD,0.9168,0.9168,1.00
mem2_ssd2_gpu1_v2_x32,g6.8xlarge_gpu1_v2,2,2,1,1,2,32,128,900,SSD,1.3952,1.1968,1.17
mem2_ssd2_gpu1_v2_x64,g6.16xlarge_gpu1_v2,2,2,1,1,2,64,256,1880,SSD,2.3552,2.2592,1.04
mem2_ssd2_gpu4_v2_x48,g6.12xlarge_gpu4_v2,2,2,1,4,2,48,192,3760,SSD,3.1872,3.1248,1.02
mem2_ssd2_gpu4_v2_x96,g6.24xlarge_gpu4_v2,2,2,1,4,2,96,384,3760,SSD,4.6272,4.4352,1.04
mem2_ssd2_gpu8_v2_x192,g6.48xlarge_gpu8_v2,2,2,1,8,2,192,768,7520,SSD,9.2544,8.1216,1.14
mem1_ssd2_v2_x2,c5.large_v2,2,2,2,0,1,2,4,159,EBS-gp3,0.0808,0.0424,1.91
mem1_ssd2_v2_x4,c5.xlarge_v2,2,2,3,0,1,4,8,318,EBS-gp2,0.1616,0.0848,1.91
mem1_ssd2_v2_x8,c5.2xlarge_v2,2,2,3,0,1,8,16,640,EBS-gp2,0.3232,0.1696,1.91
mem2_ssd2_v2_x2,m5.large_v2_ssd,2,2,3,0,2,2,8,160,EBS-gp2,0.089,0.0424,2.10
mem2_ssd2_v2_x4,m5.xlarge_v2_ssd,2,2,3,0,2,4,16,320,EBS-gp2,0.178,0.0848,2.10
mem2_ssd2_v2_x8,m5.2xlarge_v2_ssd,2,2,3,0,2,8,32,640,EBS-gp2,0.356,0.1696,2.10
mem2_ssd2_v2_x16,m5.4xlarge_v2_ssd,2,2,3,0,2,16,64,1280,EBS-gp2,0.712,0.3392,2.10
mem2_ssd2_v2_x32,m5.8xlarge_v2_ssd,2,2,3,0,2,32,128,2560,EBS-gp2,1.424,0.6784,2.10
mem2_ssd2_v2_x48,m5.12xlarge_v2_ssd,2,2,3,0,2,48,192,3840,EBS-gp2,2.136,1.0176,2.10
mem2_ssd2_v2_x64,m5.16xlarge_v2_ssd,2,2,3,0,2,64,256,5120,EBS-gp2,2.848,1.3568,2.10
mem2_ssd2_v2_x96,m5.24xlarge_v2_ssd,2,2,3,0,2,96,384,7480,EBS-gp2,4.272,2.0352,2.10
mem2_hdd2_v2_x2,m5.large_v2,2,2,4,0,2,2,8,1000,EBS-st1,0.1336,0.0898,1.49
mem2_hdd2_v2_x4,m5.xlarge_v2,2,2,4,0,2,4,16,2000,EBS-st1,0.2672,0.1796,1.49
mem3_hdd2_v2_x2,r5.large_v2,2,2,4,0,3,2,16,500,EBS-st1,0.1172,0.0532,2.20
mem3_hdd2_v2_x4,r5.xlarge_v2,2,2,4,0,3,4,32,1000,EBS-st1,0.2344,0.1064,2.20
mem3_hdd2_v2_x8,r5.2xlarge_v2,2,2,4,0,3,8,64,2000,EBS-st1,0.4688,0.2128,2.20
mem4_ssd1_x128,x1.32xlarge,1,1,1,0,4,128,1952,3840,SSD,9.1776,9.1776,1.00
mem2_ssd1_gpu_x16,g4dn.4xlarge,1,1,1,1,2,16,64,225,SSD,0.7696,0.6432,1.20
mem2_ssd1_gpu1_x32,g4dn.8xlarge_gpu1,1,1,1,1,2,32,128,900,SSD,1.3888,1.2192,1.14
mem2_ssd1_gpu1_x64,g4dn.16xlarge_gpu1,1,1,1,1,2,64,256,900,SSD,2.7776,2.4384,1.14
mem3_ssd1_gpu1_x16,gr6.4xlarge_gpu1,1,1,1,1,3,16,128,600,SSD,1.0672,0.9136,1.17
mem3_ssd1_gpu1_x32,gr6.8xlarge_gpu1,1,1,1,1,3,32,256,900,SSD,1.696,1.4528,1.17
mem2_ssd1_gpu4_x48,g4dn.12xlarge_gpu4,1,1,1,4,2,48,192,900,SSD,2.5008,2.1936,1.14
mem3_ssd1_gpu_x8,p3.2xlarge,1,1,3,1,3,8,61,160,EBS-gp2,1.984,1.9696,1.01
mem3_ssd1_gpu_x32,p3.8xlarge,1,1,3,4,3,32,244,640,EBS-gp2,7.936,7.8784,1.01
mem1_hdd1_x2,c4.large_hdd,1,1,4,0,1,2,3.75,200,EBS-st1,0.0796,0.0314,2.54
mem1_hdd1_x4,c4.xlarge_hdd,1,1,4,0,1,4,7.5,400,EBS-st1,0.1592,0.0628,2.54
mem1_hdd1_x8,c4.2xlarge_hdd,1,1,4,0,1,8,15,800,EBS-st1,0.3184,0.1256,2.54
mem1_hdd1_x16,c4.4xlarge_hdd,1,1,4,0,1,16,30,1600,EBS-st1,0.6368,0.2512,2.54
mem1_hdd1_x36,c4.8xlarge_hdd,1,1,4,0,1,36,60,3600,EBS-st1,1.4328,0.5652,2.54
mem3_ssd1_gpu_x64,p3.16xlarge,1,1,4,8,3,64,488,1280,EBS-st1,15.872,15.7568,1.01
mem3_ssd2_dragen_fpga1_x24,f2.6xlarge_dragen,1,2,1,0,3,24,256,950,SSD,2.6064,1.86,1.40
mem2_ssd2_gpu1_x4,g5.xlarge_gpu1,1,2,1,1,2,4,16,250,SSD,0.6968,0.6116,1.14
mem2_ssd2_gpu1_x8,g5.2xlarge_gpu1,1,2,1,1,2,8,32,450,SSD,0.84,0.72,1.17
mem2_ssd2_gpu1_x16,g5.4xlarge_gpu1,1,2,1,1,2,16,64,600,SSD,1.1248,1.0112,1.11
mem2_ssd2_gpu1_x32,g5.8xlarge_gpu1,1,2,1,1,2,32,128,900,SSD,1.696,1.488,1.14
mem2_ssd2_gpu1_x64,g5.16xlarge_gpu1,1,2,1,1,2,64,256,1900,SSD,2.8416,2.8416,1.00
mem2_ssd2_gpu4_x48,g5.12xlarge_gpu4,1,2,1,4,2,48,192,3800,SSD,3.9312,3.9312,1.00
mem2_ssd2_gpu4_x96,g5.24xlarge_gpu4,1,2,1,4,2,96,384,3800,SSD,5.6448,5.6448,1.00
mem2_ssd2_gpu8_x192,g5.48xlarge_gpu8,1,2,1,8,2,192,768,7600,SSD,11.2896,11.2896,1.00
mem2_ssd2_x2,m4.large_v2_ssd,1,2,3,0,2,2,8,160,EBS-gp2,0.089,0.0424,2.10
mem2_ssd2_x4,m4.xlarge_v2_ssd,1,2,3,0,2,4,16,320,EBS-gp2,0.178,0.0848,2.10
mem2_ssd2_x8,m4.2xlarge_v2_ssd,1,2,3,0,2,8,32,640,EBS-gp2,0.356,0.1696,2.10
mem2_ssd2_x16,m4.4xlarge_v2_ssd,1,2,3,0,2,16,64,1280,EBS-gp2,0.712,0.3392,2.10
mem2_ssd2_x40,m4.10xlarge_v2_ssd,1,2,3,0,2,40,160,3200,EBS-gp2,1.78,0.848,2.10
mem2_ssd2_x64,m4.16xlarge_v2_ssd,1,2,3,0,2,64,256,5120,EBS-gp2,2.848,1.3568,2.10
mem3_ssd3_x2,i3en.large,1,3,1,0,3,2,16,1250,SSD,0.1436,0.1348,1.07
mem3_ssd3_x4,i3en.xlarge,1,3,1,0,3,4,32,2500,SSD,0.2872,0.2696,1.07
mem3_ssd3_x8,i3en.2xlarge,1,3,1,0,3,8,64,5000,SSD,0.5744,0.5392,1.07
mem3_ssd3_x12,i3en.3xlarge,1,3,1,0,3,12,96,7500,SSD,0.8616,0.8088,1.07
mem3_ssd3_x24,i3en.6xlarge,1,3,1,0,3,24,192,15000,SSD,1.7232,1.6176,1.07
mem3_ssd3_x48,i3en.12xlarge,1,3,1,0,3,48,384,30000,SSD,3.4464,3.2352,1.07
mem3_ssd3_x96,i3en.24xlarge,1,3,1,0,3,96,768,60000,SSD,6.8928,6.4704,1.07

     

    storage_rank is =XLOOKUP(K2, {"SSD","EBS-gp3","EBS-gp2","EBS-st1"}, {1,2,3,4})

     

    1

Please sign in to leave a comment.