CyberGIS-Compute

Journal Articles

2024

1. SoftwareX

   CyberGIS-Compute: Middleware for democratizing scalable geocomputation

   Michels, Alexander C., Padmanabhan, Anand, Xiao, Zimo, Kotak, Mit, Baig, Furqan, and Wang, Shaowen

   SoftwareX, 2024

   Abs DOI

   CyberGIS—geographic information science and systems (GIS) based on advanced cyberinfrastructure—is becoming increasingly important to tackling a variety of socio-environmental problems like climate change, disaster management, and water security. While recent advances in high-performance computing (HPC) have the potential to help address these problems, the technical knowledge required to use HPC has posed challenges to many domain experts. In this paper, we present CyberGIS-Compute: a geospatial middleware tool designed to democratize HPC access for solving diverse socio-environmental problems. CyberGIS-Compute does this by providing a simple user interface in Jupyter, streamlining the process of integrating domain-specific models with HPC, and establishing a suite of APIs friendly to domain experts.

Conference Proceedings

2024

1. PEARC

   Providing Accessible Software Environments Across Science Gateways and HPC

   Michels, Alexander, Kotak, Mit, Padmanabhan, Anand, Speaks, John, and Wang, Shaowen

   Practice and Experience in Advanced Research Computing 2024: Human Powered Computing, 2024

   Abs DOI

   While High-Performance Computing (HPC) resources are powerful for tackling complex, computationally intensive analysis and modeling problems, access to these resources varies across disciplines. Domain scientists in a variety of fields such as social and environmental sciences often lack in-depth technical skills (e.g., familiarity with terminal, knowledge of job schedulers) to effectively utilize HPC resources, hindering desired research. In this context, CyberGIS-Compute is a middleware toolkit designed to democratize HPC access with the main goal of enabling domain scientists in diverse fields to solve computationally intensive problems. A key challenge facing model developers on CyberGIS-Compute is to create a containerized software environment for their models. Domain experts unfamiliar with HPC are generally unfamiliar with containerization technologies (e.g., Docker, Singularity) and thus unable to create/test containers to execute their models. But if they have access to science gateways, they would want to use these familiar software environments on HPC resources. This paper describes a novel approach to integrating the Cern Virtual Machine File System (CVMFS) into CyberGIS-Compute to provide consistent software environments across science gateways and HPC resources.

2022

1. PEARC

   CyberGIS-Cloud: A Unified Middleware Framework for Cloud-Based Geospatial Research and Education

   Baig, Furqan, Michels, Alexander, Xiao, Zimo, Han, Su Yeon, Padmanabhan, Anand, Li, Zhiyu, and Wang, Shaowen

   Practice and Experience in Advanced Research Computing, 2022

   Abs DOI

   Interest in cloud-based cyberinfrastructure continues to grow within the geospatial community to tackle contemporary big data challenges. Distributed computing frameworks, deployed over the cloud, provide scalable and low-maintenance solutions to accelerate geospatial research and education. However, for scientists and researchers, the usage of such resources is highly constrained by the steep curve for learning diverse sets of platform-specific tools and APIs. This paper presents CyberGIS-Cloud as a unified middleware to streamline the execution of distributed geospatial workflows over multiple cloud backends with easy-to-use interfaces. CyberGIS-Cloud employs bringing computation-to-data model by abstracting and automating job execution over distributed resources hosted in the cloud environment where the data resides. We present details of CyberGIS-Cloud with support for popular distributed computing frameworks backed by research-oriented JetStream Cloud and commercial Google Cloud Platform.

2021

1. SIGSPATIAL SpatialAPI

   CyberGIS-Compute for Enabling Computationally Intensive Geospatial Research (Ext. Abs.)

   Padmanabhan, Anand, Xiao, Zimo, Vandewalle, Rebecca, Baig, Furqan, Michels, Alexander, Li, Zhiyu, and Wang, Shaowen

   SpatialAPI’21: Proceedings of the 3rd ACM SIGSPATIAL International Workshop on APIs and Libraries for Geospatial Data Science, 2021

   Abs DOI

   In this tutorial, we will first start with the basics of CyberGISJupyter and CyberGIS-Compute, then introduce the Python SDK for CyberGIS-Compute with a simple Hello World example. Then, we will take multiple real-world geospatial applications use-cases like spatial accessibility and wildfire evacuation simulation using agent based modeling. We will also provide pointers on how to contribute applications to the CyberGIS-Compute framework.

2. Gateways

   Enabling Computationally Intensive Geospatial Research on CyberGIS-Jupyter with CyberGIS-Compute (Ext. Abs.)

   Padmanabhan, Anand, Xiao, Zimo, Vandewalle, Rebecca, Michels, Alexander, and Wang, Shaowen

   Proceedings of Gateways 2021, 2021

   Abs DOI

   Geospatial research and education have become increasingly dependent on cyberGIS, defined as geographic information science and systems based on advanced cyberinfrastructure (CI), [1] to tackle computation and data challenges. However, the use of advanced cyberGIS capabilities has typically been constrained to a small set of research groups who have the technical expertise of using CI resources. Over the past few years CyberGIS-Jupyter [2,3] has been developed to provide access to cyberGIS capabilities through an easy-to-use Jupyter Notebook interface which has made cyberGIS more accessible. For many cyberGIS and geospatial applications accessing CI resources needed for solving complex problems at scale. However, leveraging CI resources for geospatial application is challenging both due to the steep learning curve and lack of appropriate tools. CyberGIS-Compute fills this gap by providing an easy-to-use middleware tool for using and contributing geospatial application codes that leverage CI resources. This substantially lowers the learning curve for both geospatial users and developers to access cyberGIS capabilities at scale. CyberGIS-Compute is backed by Virtual ROGER (Resourcing Open Geospatial Education and Research); a geospatial supercomputer with access to a number of readily available popular geospatial libraries. With CyberGIS-Compute we have designed an easy-to-use middleware and associated Python SDK to provide access to CyberGIS capabilities, allowing geospatial applications to easily scale and employ advanced cyberinfrastructure resources. This presentation will first describe the basics of CyberGIS-Jupyter and CyberGIS-Compute, then introduce the Python SDK for CyberGIS-Compute with a simple example. Then, we will take multiple real-world geospatial applications use-cases like spatial accessibility and wildfire evacuation simulation using agent based modeling. Lastly, we will also descrive mechanism to contribute applications to the CyberGIS-Compute framework.