Return to search results
Active Evaluation Software for Selection of Ground Truth Labels
This software repository contains a python package Aegis (Active Evaluator Germane Interactive Selector) package that allows us to evaluate machine learning systems's performance (according to a metric such as accuracy) by adaptively sampling trials to label from an unlabeled test set to minimize the number of labels needed. This includes sample (public) data as well as a simulation script that tests different label-selecting strategies on already labelled test sets. This software is configured so that users can add their own data and system outputs to test evaluation.
Resources
1 resource available
-
DOI Access for Active Evaluation Software for Selection of Ground Truth Labels
FILE
Find Related Datasets
Search by Tags
Click any tag below to search for similar datasets
Complete Metadata
| @type | dcat:Dataset |
|---|---|
| accessLevel | public |
| bureauCode |
[ "006:55" ] |
| contactPoint |
{ "fn": "Peter Fontana", "hasEmail": "mailto:peter.fontana@nist.gov" } |
| description | This software repository contains a python package Aegis (Active Evaluator Germane Interactive Selector) package that allows us to evaluate machine learning systems's performance (according to a metric such as accuracy) by adaptively sampling trials to label from an unlabeled test set to minimize the number of labels needed. This includes sample (public) data as well as a simulation script that tests different label-selecting strategies on already labelled test sets. This software is configured so that users can add their own data and system outputs to test evaluation. |
| distribution |
[ { "title": "DOI Access for Active Evaluation Software for Selection of Ground Truth Labels", "accessURL": "https://doi.org/10.18434/M32227" } ] |
| identifier | ark:/88434/mds2-2227 |
| issued | 2020-07-09 |
| keyword |
[ "active evaluation", "ar", "machine learning" ] |
| landingPage | https://github.com/usnistgov/active-evaluation |
| language |
[ "en" ] |
| license | https://www.nist.gov/open/license |
| modified | 2020-04-28 00:00:00 |
| programCode |
[ "006:045" ] |
| publisher |
{ "name": "National Institute of Standards and Technology", "@type": "org:Organization" } |
| theme |
[ "Information Technology:Data and informatics" ] |
| title | Active Evaluation Software for Selection of Ground Truth Labels |
