Unit and Regression Tests of Scientific Software: A Study on SWMM
Testing helps assure software quality by executing programs and uncovering bugs. Scientific software developers often find it challenging to carry out systematic and automated testing due to reasons such as inherent model uncertainties and complex floating-point computations. Extending the recent work on analyzing the unit tests written by the developers of the Storm Water Management Model (SWMM), we report in this paper the investigation of both unit and regression tests of SWMM. The results show that the 1,458 SWMM tests have a 54.0% code coverage and an 82.4% User’s Manual coverage. Meanwhile, an examination of eight regression tests from a test set shows a 79.5% code coverage and a near 100% User’s Manual coverage. We also observe a “getter-setter-getter” testing pattern from the SWMM unit tests and suggest a diversified way of designing or adopting regression tests.
This dataset is associated with the following publication:
Peng, Z., X. Lin, M. Simon, and N. Niu. Unit and Regression Tests of Scientific Software: A Study on SWMM. Journal of Computational Science. Elsevier B.V., Amsterdam, NETHERLANDS, 53: 101347, (2021).
Resources
3 resources available
-
Mappings_of_SWMM_unit_tests.xlsx
APPLICATION/VND.OPENXMLFORMATS-OFFICEDOCUMENT.SPREADSHEETML.SHEET -
regression test data.xlsx
APPLICATION/VND.OPENXMLFORMATS-OFFICEDOCUMENT.SPREADSHEETML.SHEET -
test study on swmm(data table).xlsx
APPLICATION/VND.OPENXMLFORMATS-OFFICEDOCUMENT.SPREADSHEETML.SHEET
Find Related Datasets
Search by Tags
Click any tag below to search for similar datasets
Complete Metadata
| @type | dcat:Dataset |
|---|---|
| accessLevel | public |
| bureauCode |
[ "020:00" ] |
| contactPoint |
{ "fn": "Michelle Simon", "hasEmail": "mailto:simon.michelle@epa.gov" } |
| describedBy | https://pasteur.epa.gov/uploads/10.23719/1519338/documents/Unit%20and%20Regression%20Tests%20of%20Scientific%20Softwaredatadictionary.docx |
| describedByType | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| description | Testing helps assure software quality by executing programs and uncovering bugs. Scientific software developers often find it challenging to carry out systematic and automated testing due to reasons such as inherent model uncertainties and complex floating-point computations. Extending the recent work on analyzing the unit tests written by the developers of the Storm Water Management Model (SWMM), we report in this paper the investigation of both unit and regression tests of SWMM. The results show that the 1,458 SWMM tests have a 54.0% code coverage and an 82.4% User’s Manual coverage. Meanwhile, an examination of eight regression tests from a test set shows a 79.5% code coverage and a near 100% User’s Manual coverage. We also observe a “getter-setter-getter” testing pattern from the SWMM unit tests and suggest a diversified way of designing or adopting regression tests. This dataset is associated with the following publication: Peng, Z., X. Lin, M. Simon, and N. Niu. Unit and Regression Tests of Scientific Software: A Study on SWMM. Journal of Computational Science. Elsevier B.V., Amsterdam, NETHERLANDS, 53: 101347, (2021). |
| distribution |
[ { "title": "Mappings_of_SWMM_unit_tests.xlsx", "mediaType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", "downloadURL": "https://pasteur.epa.gov/uploads/10.23719/1519338/Mappings_of_SWMM_unit_tests.xlsx" }, { "title": "regression test data.xlsx", "mediaType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", "downloadURL": "https://pasteur.epa.gov/uploads/10.23719/1519338/regression%20test%20data.xlsx" }, { "title": "test study on swmm(data table).xlsx", "mediaType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", "downloadURL": "https://pasteur.epa.gov/uploads/10.23719/1519338/test%20study%20on%20swmm%28data%20table%29.xlsx" } ] |
| identifier | https://doi.org/10.23719/1519338 |
| keyword |
[ "Storm Water Management Model (SWMM)", "User’s Manual", "regression testing", "scientific software", "test-coverage", "unit testing" ] |
| license | https://pasteur.epa.gov/license/sciencehub-license.html |
| modified | 2020-09-08 |
| programCode |
[ "020:096" ] |
| publisher |
{ "name": "U.S. EPA Office of Research and Development (ORD)", "subOrganizationOf": { "name": "U.S. Environmental Protection Agency", "subOrganizationOf": { "name": "U.S. Government" } } } |
| references |
[ "https://doi.org/10.1016/j.jocs.2021.101347", "https://pasteur.epa.gov/uploads/10.23719/1519338/documents/SWMM%20OpenCppCoverage%20Step.pdf" ] |
| rights |
null
|
| title | Unit and Regression Tests of Scientific Software: A Study on SWMM |
