SparkRS - Spark for Remote Sensing, Phase I

Metadata Updated: November 12, 2020

The proposed innovation is Spark-RS, an open source software project that enables GPU-accelerated remote sensing workflows in an Apache Spark distributed computing cluster. Current state-of-the-art parallel systems like Hadoop and Spark offer horizontally scalable analytics and reduced costs for enterprises, but weren't built to natively consume and process large remote sensing raster datasets. Conversely, GPUs can vastly accelerate image processing operations. Some open source projects have arisen that showcase hybrid Hadoop/GPU computing. However, there are no mature open source projects that utilize GPUs within Spark (an eventual replacement of MapReduce) and none that were built to process large remote sensing imagery. This is the primary role of the proposed innovation, Spark-RS.

Spark-RS contains three primary components. One is a parallel large image loading component that quickly loads large multi-band imagery into a Spark cluster. The second component is a remote sensing library for Spark applications. It provides an API for reading and writing large images and wraps many common image operations from existing open source and NASA-built remote sensing libraries. The third component is a GPU management library for Spark. It simplifies and abstracts utilization of GPUs within a Spark application.

Access & Use Information

Public: This dataset is intended for public access and use. License: No license information was provided. If this work was prepared by an officer or employee of the United States government as part of that person's official duties it is considered a U.S. Government Work.

Downloads & Resources


Metadata Created Date November 12, 2020
Metadata Updated Date November 12, 2020

Metadata Source

Harvested from NASA Data.json

Additional Metadata

Resource Type Dataset
Metadata Created Date November 12, 2020
Metadata Updated Date November 12, 2020
Publisher Space Technology Mission Directorate
Unique Identifier Unknown
Identifier TECHPORT_33623
Data First Published 2015-12-01
Data Last Modified 2020-01-29
Public Access Level public
Bureau Code 026:00
Metadata Context
Metadata Catalog ID
Schema Version
Catalog Describedby
Harvest Object Id 83ce9140-8644-40d7-ac4b-22fbc5387953
Harvest Source Id 58f92550-7a01-4f00-b1b2-8dc953bd598f
Harvest Source Title NASA Data.json
Homepage URL
Program Code 026:027
Source Datajson Identifier True
Source Hash ba57f6a46597f1daf7c69c5f92d5ced178c31a3f
Source Schema Version 1.1

Didn't find what you're looking for? Suggest a dataset here.