A Framework for Systematic Benchmarking of Monitoring and Diagnostic Systems

Metadata Updated: August 1, 2018

In this paper, we present an architecture and a formal framework to be used for systematic benchmarking of monitoring and diagnostic systems and for producing comparable performance assessments of different diagnostic technologies. The framework defines a number of standardized specifications, which include a fault catalog, a library of modular test scenarios, and a common protocol for gathering and processing diagnostic data. At the center of the framework are 13 benchmarking metric definitions. The calculation of metrics is illustrated on a probabilistic model-based diagnosis algorithm utilizing Bayesian reasoning techniques. The diagnosed system is a real-world electrical power system, namely the Advanced Diagnostics and Prognostics Testbed (ADAPT) developed and located at the NASA Ames Research Center. The proposed benchmarking approach shows how to generate realistic diagnostic data sets for large-scale, complex engineering systems, and how the generated experimental data can be used to enable “apples to apples” assessments of the effectiveness of different diagnostic and monitoring algorithms.

Access & Use Information

Public: This dataset is intended for public access and use. License: U.S. Government Work

Downloads & Resources


Metadata Created Date August 1, 2018
Metadata Updated Date August 1, 2018
Data Update Frequency irregular

Metadata Source

Harvested from NASA Data.json

Additional Metadata

Resource Type Dataset
Metadata Created Date August 1, 2018
Metadata Updated Date August 1, 2018
Publisher Dashlink
Unique Identifier DASHLINK_858
Maintainer Email
Id {$oid: 56cf5b00a759fdadc44e5809}
Public Access Level public
Data Update Frequency irregular
Bureau Code 026:00
Metadata Context https://project-open-data.cio.gov/v1.1/schema/catalog.jsonld
Schema Version https://project-open-data.cio.gov/v1.1/schema
Catalog Describedby https://project-open-data.cio.gov/v1.1/schema/catalog.json
Harvest Object Id 05f81c4d-4f0b-4039-a9dc-9ab3ef895224
Harvest Source Id 39e4ad2a-47ca-4507-8258-852babd0fd99
Harvest Source Title NASA Data.json
Data First Published 2013-12-12T17:29:19
Homepage URL https://c3.nasa.gov/dashlink/resources/858/
Language en-US
License http://www.usa.gov/publicdomain/label/1.0/
Data Last Modified 2013-12-12T17:30:31
Program Code 026:029
Publisher Hierarchy U.S. Government > National Aeronautics and Space Administration > Dashlink
Source Datajson Identifier True
Source Hash 69f4426a8016b8534076c7f029e1583392295c40
Source Schema Version 1.1

Didn't find what you're looking for? Suggest a dataset here.