Skip to main content
U.S. flag

An official website of the United States government

Challenging Medically-Relevant Genes Benchmark Set

Published by National Institute of Standards and Technology | Department of Commerce | Catalog Last Checked: October 02, 2026 at 07:52 PM | Dataset Last Updated: September 29, 2021
CMRG v1.00 of a small variant benchmark and structural variant benchmark focused on 273 challenging medically relevant genes for the Genome in a Bottle (GIAB) sample HG002 (aka Ashkenazi son). These benchmarks were generated from a trio-based hifiasm v0.11 (https://doi.org/10.1038/s41592-020-01056-5) diploid assembly of HG002 using PacBio HiFi reads for HG002 for assembly and partitioning into phased haplotypes using Illumina reads for the parents, HG003 and HG004. This benchmark contains vcfs for small and structural variants along with corresponding benchmark bed files indicating regions that are homozygous reference if they do not have a variant in the vcf. We extensively curated the variant calls, excluding any found to be questionable or errors. This benchmark helps measure performance in important challenging regions, including challenging segmental duplications, regions with complex variants, regions with structural variants, and regions affected by false duplications in GRCh37 or GRCh38. This benchmark is described in https://doi.org/10.1101/2021.06.07.444885.

Resources

65 resources available

Find Related Datasets

Search by Tags

Click any tag below to search for similar datasets

Complete Metadata
@type dcat:Dataset
accessLevel public
bureauCode 
[
  "006:55"
]
contactPoint 
{
  "fn": "Nathanael David Olson",
  "hasEmail": "mailto:nathanael.olson@nist.gov"
}
description CMRG v1.00 of a small variant benchmark and structural variant benchmark focused on 273 challenging medically relevant genes for the Genome in a Bottle (GIAB) sample HG002 (aka Ashkenazi son). These benchmarks were generated from a trio-based hifiasm v0.11 (https://doi.org/10.1038/s41592-020-01056-5) diploid assembly of HG002 using PacBio HiFi reads for HG002 for assembly and partitioning into phased haplotypes using Illumina reads for the parents, HG003 and HG004. This benchmark contains vcfs for small and structural variants along with corresponding benchmark bed files indicating regions that are homozygous reference if they do not have a variant in the vcf. We extensively curated the variant calls, excluding any found to be questionable or errors. This benchmark helps measure performance in important challenging regions, including challenging segmental duplications, regions with complex variants, regions with structural variants, and regions affected by false duplications in GRCh37 or GRCh38. This benchmark is described in https://doi.org/10.1101/2021.06.07.444885.
distribution 
[
  {
    "title": "DOI Access for Challenging Medically-Relevant Genes Benchmark Set",
    "accessURL": "https://doi.org/10.18434/mds2-2475"
  },
  {
    "title": "GIAB FTP Site",
    "accessURL": "https://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/release/AshkenazimTrio/HG002_NA24385_son/CMRG_v1.00/",
    "description": "NCBI Hosted Genome In A Bottle FTP Site"
  },
  {
    "title": "Code for Manuscript Analysis Repository",
    "accessURL": "https://github.com/usnistgov/cmrg-benchmarkset-manuscript",
    "description": "Github repository with code used to generate figures and perform analysis for manuscript."
  },
  {
    "title": "Code Repository",
    "accessURL": "https://github.com/usnistgov/giab-cmrg-benchmarkset",
    "description": "Github repository with code used to generate benchmark sets."
  },
  {
    "mediaType": "text/plain",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/README.md"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SmallVariant/HG002_CHM13v1.0_CMRG_smallvar_v1.00_draft.bed"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SmallVariant/HG002_CHM13v1.0_CMRG_smallvar_v1.00_draft.vcf.gz"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SmallVariant/HG002_CHM13v1.0_CMRG_smallvar_v1.00_draft.vcf.gz.tbi"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SupplementaryFiles/HG002_CHM13_CMRG_smallvar_v1.00_GRCh38-equiv-regions_draft.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SupplementaryFiles/HG002v11-align2-CHM13v1.0/HG002v11-align2-CHM13v1.0.dip.bed"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SupplementaryFiles/HG002v11-align2-CHM13v1.0/HG002v11-align2-CHM13v1.0.dip.vcf.gz"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SupplementaryFiles/HG002v11-align2-CHM13v1.0/HG002v11-align2-CHM13v1.0.dip.vcf.gz.tbi"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SupplementaryFiles/HG002v11-align2-CHM13v1.0/HG002v11-align2-CHM13v1.0.hap1.bam"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SupplementaryFiles/HG002v11-align2-CHM13v1.0/HG002v11-align2-CHM13v1.0.hap1.bam.bai"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SupplementaryFiles/HG002v11-align2-CHM13v1.0/HG002v11-align2-CHM13v1.0.hap2.bam"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/CHM13v1.0/SupplementaryFiles/HG002v11-align2-CHM13v1.0/HG002v11-align2-CHM13v1.0.hap2.bam.bai"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SmallVariant/HG002_GRCh37_CMRG_smallvar_v1.00.bed"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SmallVariant/HG002_GRCh37_CMRG_smallvar_v1.00.vcf.gz"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SmallVariant/HG002_GRCh37_CMRG_smallvar_v1.00.vcf.gz.tbi"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/StructuralVariant/HG002_GRCh37_CMRG_SV_v1.00.bed"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/StructuralVariant/HG002_GRCh37_CMRG_SV_v1.00.vcf.gz"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/StructuralVariant/HG002_GRCh37_CMRG_SV_v1.00.vcf.gz.tbi"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SupplementaryFiles/GRCh37_CMRG_benchmark_gene_coordinates.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SupplementaryFiles/HG002v11-align2-GRCh37/HG002v11-align2-GRCh37.dip.bed"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SupplementaryFiles/HG002v11-align2-GRCh37/HG002v11-align2-GRCh37.dip.vcf.gz"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SupplementaryFiles/HG002v11-align2-GRCh37/HG002v11-align2-GRCh37.dip.vcf.gz.tbi"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SupplementaryFiles/HG002v11-align2-GRCh37/HG002v11-align2-GRCh37.hap1.bam"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SupplementaryFiles/HG002v11-align2-GRCh37/HG002v11-align2-GRCh37.hap1.bam.bai"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SupplementaryFiles/HG002v11-align2-GRCh37/HG002v11-align2-GRCh37.hap2.bam"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh37/SupplementaryFiles/HG002v11-align2-GRCh37/HG002v11-align2-GRCh37.hap2.bam.bai"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SmallVariant/HG002_GRCh38_CMRG_smallvar_v1.00.bed"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SmallVariant/HG002_GRCh38_CMRG_smallvar_v1.00.vcf.gz"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SmallVariant/HG002_GRCh38_CMRG_smallvar_v1.00.vcf.gz.tbi"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/StructuralVariant/HG002_GRCh38_CMRG_SV_v1.00.bed"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/StructuralVariant/HG002_GRCh38_CMRG_SV_v1.00.vcf.gz"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/StructuralVariant/HG002_GRCh38_CMRG_SV_v1.00.vcf.gz.tbi"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SupplementaryFiles/GRCh38_CMRG_benchmark_gene_coordinates.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SupplementaryFiles/HG002v11-align2-GRCh38/HG002v11-align2-GRCh38.dip.bed"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SupplementaryFiles/HG002v11-align2-GRCh38/HG002v11-align2-GRCh38.dip.vcf.gz"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SupplementaryFiles/HG002v11-align2-GRCh38/HG002v11-align2-GRCh38.dip.vcf.gz.tbi"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SupplementaryFiles/HG002v11-align2-GRCh38/HG002v11-align2-GRCh38.hap1.bam"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SupplementaryFiles/HG002v11-align2-GRCh38/HG002v11-align2-GRCh38.hap1.bam.bai"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SupplementaryFiles/HG002v11-align2-GRCh38/HG002v11-align2-GRCh38.hap2.bam"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/benchmark_sets/GRCh38/SupplementaryFiles/HG002v11-align2-GRCh38/HG002v11-align2-GRCh38.hap2.bam.bai"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/chksum.md5"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh37_MRG_GAPs.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh37_curation_medicalgene_SV_errorsorunsure.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh37_curation_medicalgene_smallvar_complexrepeat_errorsorunsure_repeatexpanded.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh37_hifiasm_error.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh37_mrg_full_gene.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh38_CD4_gaps.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh38_CD4_gaps_slop50.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh38_MRG_GAPs.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh38_curation_medicalgene_SV_errorsorunsure_repeatexpanded.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh38_curation_medicalgene_smallvar_complexrepeat_errorsorunsure_repeatexpanded.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh38_hifiasm_error.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/GRCh38_mrg_full_gene.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/HiCanu_2.1_HG002_GRCh37_difficult_medical_gene_smallvar_benchmark_v0.02.03_intersected_FPs_repeatexpanded_slop50.bed"
  },
  {
    "mediaType": "application/octet-stream",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/HiCanu_2.1_HG002_GRCh38_difficult_medical_gene_smallvar_benchmark_v0.02.03_intersected_FPs_repeatexpanded_slop50.bed"
  },
  {
    "mediaType": "text/plain",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/HiCanu_2.1_HG002_GRCh38_difficult_medical_gene_smallvar_benchmark_v0.02.03_intersected_subtract_FPs_repeatexpanded_slop50_manual_curation_sites.tsv_manual_curation_sites.txt"
  },
  {
    "mediaType": "text/tab-separated-values",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/dependencies/combined%20curation%20responses%20from%20benchmarking%20with%20sm%20variant%20v0.02.03%20-%20GRCh37andGRCh38.tsv"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/hifiasm-assembly/HG002-v0.11.mat.fa.gz"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/hifiasm-assembly/HG002-v0.11.mat.gff.gz"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/hifiasm-assembly/HG002-v0.11.pat.fa.gz"
  },
  {
    "mediaType": "application/gzip",
    "downloadURL": "https://data.nist.gov/od/ds/ark:/88434/mds2-2475/hifiasm-assembly/HG002-v0.11.pat.gff.gz"
  }
]
identifier ark:/88434/mds2-2475
keyword 
[
  "Bioinformatics",
  "Bioinformatics",
  "DNA sequencing",
  "Human genomics",
  "Medical genomics",
  "Reference materials"
]
landingPage https://data.nist.gov/od/id/mds2-2475
language 
[
  "en"
]
license https://www.nist.gov/open/license
modified 2021-09-29 00:00:00
programCode 
[
  "006:045"
]
publisher 
{
  "name": "National Institute of Standards and Technology",
  "@type": "org:Organization"
}
references 
[
  "https://doi.org/10.1038/s41592-020-01056-5",
  "https://doi.org/10.1101/2021.06.07.444885"
]
theme 
[
  "Bioscience:Genomics"
]
title Challenging Medically-Relevant Genes Benchmark Set

data.gov

An official website of the GSA's Technology Transformation Services

Looking for U.S. government information and services?
Visit USA.gov