NIST GenAI Code Pilot Evaluation: Scoring Software
NIST Generative AI (GenAI) is a new evaluation program administered by the NIST Information Technology Laboratory to assess generative AI technologies developed by the research community from around the world. NIST GenAI is an umbrella program that supports various evaluations for research and measurement science in Generative AI by providing a platform for Test and Evaluation. One component of GenAI is NIST GenAI Code, which currently evaluates: Can AI generate code for testing software reliably? This item is the open-source scoring code software used to score NIST GenAI Code submissions. The GenAI Code Pilot Software is open-sourced and available on Github under the USNISTGOV Organization, repository name "test_code_eval".
Resources
3 resources available
-
NIST GenAI Code Pilot Evaluation:
FILE -
NIST GenAI Main Site
FILE -
Scoring Software: test_code_eval Github Repository
GITH
Find Related Datasets
Search by Tags
Click any tag below to search for similar datasets
Complete Metadata
| @type | dcat:Dataset |
|---|---|
| accessLevel | public |
| bureauCode |
[ "006:55" ] |
| contactPoint |
{ "fn": "Peter Fontana", "hasEmail": "mailto:peter.fontana@nist.gov" } |
| description | NIST Generative AI (GenAI) is a new evaluation program administered by the NIST Information Technology Laboratory to assess generative AI technologies developed by the research community from around the world. NIST GenAI is an umbrella program that supports various evaluations for research and measurement science in Generative AI by providing a platform for Test and Evaluation. One component of GenAI is NIST GenAI Code, which currently evaluates: Can AI generate code for testing software reliably? This item is the open-source scoring code software used to score NIST GenAI Code submissions. The GenAI Code Pilot Software is open-sourced and available on Github under the USNISTGOV Organization, repository name "test_code_eval". |
| distribution |
[ { "title": "NIST GenAI Code Pilot Evaluation:", "accessURL": "https://ai-challenges.nist.gov/code", "description": "Link to NIST GenAI Code Pilot Evaluation" }, { "title": "NIST GenAI Main Site", "accessURL": "https://ai-challenges.nist.gov/genai", "description": "Main URL for the NIST Generative AI (GenAI) Program" }, { "title": "Scoring Software: test_code_eval Github Repository", "format": "github repository", "accessURL": "https://github.com/usnistgov/test_code_eval", "description": "github repository "test_code_eval" with software code" } ] |
| identifier | ark:/88434/mds2-3858 |
| issued | 2025-07-17 |
| keyword |
[ "artificial intelligence", "code testing", "evaluating test code", "generative AI", "software testing", "test code" ] |
| landingPage | https://data.nist.gov/od/id/mds2-3858 |
| language |
[ "en" ] |
| license | https://www.nist.gov/open/license |
| modified | 2025-06-06 00:00:00 |
| programCode |
[ "006:045" ] |
| publisher |
{ "name": "National Institute of Standards and Technology", "@type": "org:Organization" } |
| theme |
[ "Information Technology:Software research" ] |
| title | NIST GenAI Code Pilot Evaluation: Scoring Software |
