The concept behind CyberGIS-Compute
CyberGIS-Compute is a tool for running code on HPC from Jupyter notebooks. Compute is made up of two parts: (1) the Core which interacts with HPC resources and provides a RESTful API and (2) the SDK which provides users and developers a Python interface. It is specifically geared towards geospatial development and integrated into the CyberGISX and CJW ecosystem, but general enough that anyone can deploy and use it!
For more info see the CyberGIS-Compute Python SDK webpage.
Associated Publications
Journal Articles
2024
CyberGIS-Compute: Middleware for democratizing scalable geocomputation
Michels, Alexander C.,
Padmanabhan, Anand,
Xiao, Zimo,
Kotak, Mit,
Baig, Furqan,
and
Wang, Shaowen
SoftwareX,
2024
CyberGIS—geographic information science and systems (GIS) based on advanced cyberinfrastructure—is becoming increasingly important to tackling a variety of socio-environmental problems like climate change, disaster management, and water security. While recent advances in high-performance computing (HPC) have the potential to help address these problems, the technical knowledge required to use HPC has posed challenges to many domain experts. In this paper, we present CyberGIS-Compute: a geospatial middleware tool designed to democratize HPC access for solving diverse socio-environmental problems. CyberGIS-Compute does this by providing a simple user interface in Jupyter, streamlining the process of integrating domain-specific models with HPC, and establishing a suite of APIs friendly to domain experts.
Conference Proceedings
2022
CyberGIS-Cloud: A Unified Middleware Framework for Cloud-Based Geospatial Research and Education
Baig, Furqan,
Michels, Alexander,
Xiao, Zimo,
Han, Su Yeon,
Padmanabhan, Anand,
Li, Zhiyu,
and
Wang, Shaowen
Practice and Experience in Advanced Research Computing,
2022
Interest in cloud-based cyberinfrastructure continues to grow within the geospatial community to tackle contemporary big data challenges. Distributed computing frameworks, deployed over the cloud, provide scalable and low-maintenance solutions to accelerate geospatial research and education. However, for scientists and researchers, the usage of such resources is highly constrained by the steep curve for learning diverse sets of platform-specific tools and APIs. This paper presents CyberGIS-Cloud as a unified middleware to streamline the execution of distributed geospatial workflows over multiple cloud backends with easy-to-use interfaces. CyberGIS-Cloud employs bringing computation-to-data model by abstracting and automating job execution over distributed resources hosted in the cloud environment where the data resides. We present details of CyberGIS-Cloud with support for popular distributed computing frameworks backed by research-oriented JetStream Cloud and commercial Google Cloud Platform.
2021
CyberGIS-Compute for Enabling Computationally Intensive Geospatial Research (Ext. Abs.)
Padmanabhan, Anand,
Xiao, Zimo,
Vandewalle, Rebecca,
Baig, Furqan,
Michels, Alexander,
Li, Zhiyu,
and
Wang, Shaowen
SpatialAPI’21: Proceedings of the 3rd ACM SIGSPATIAL International Workshop on APIs and Libraries for Geospatial Data Science,
2021
In this tutorial, we will first start with the basics of CyberGISJupyter and CyberGIS-Compute, then introduce the Python SDK for CyberGIS-Compute with a simple Hello World example. Then, we will take multiple real-world geospatial applications use-cases like spatial accessibility and wildfire evacuation simulation using agent based modeling. We will also provide pointers on how to contribute applications to the CyberGIS-Compute framework.
Enabling Computationally Intensive Geospatial Research on CyberGIS-Jupyter with CyberGIS-Compute (Ext. Abs.)
Proceedings of Gateways 2021,
2021
Geospatial research and education have become increasingly dependent on cyberGIS, defined as geographic information science and systems based on advanced cyberinfrastructure (CI), [1] to tackle computation and data challenges. However, the use of advanced cyberGIS capabilities has typically been constrained to a small set of research groups who have the technical expertise of using CI resources. Over the past few years CyberGIS-Jupyter [2,3] has been developed to provide access to cyberGIS capabilities through an easy-to-use Jupyter Notebook interface which has made cyberGIS more accessible. For many cyberGIS and geospatial applications accessing CI resources needed for solving complex problems at scale. However, leveraging CI resources for geospatial application is challenging both due to the steep learning curve and lack of appropriate tools. CyberGIS-Compute fills this gap by providing an easy-to-use middleware tool for using and contributing geospatial application codes that leverage CI resources. This substantially lowers the learning curve for both geospatial users and developers to access cyberGIS capabilities at scale. CyberGIS-Compute is backed by Virtual ROGER (Resourcing Open Geospatial Education and Research); a geospatial supercomputer with access to a number of readily available popular geospatial libraries.
With CyberGIS-Compute we have designed an easy-to-use middleware and associated Python SDK to provide access to CyberGIS capabilities, allowing geospatial applications to easily scale and employ advanced cyberinfrastructure resources. This presentation will first describe the basics of CyberGIS-Jupyter and CyberGIS-Compute, then introduce the Python SDK for CyberGIS-Compute with a simple example. Then, we will take multiple real-world geospatial applications use-cases like spatial accessibility and wildfire evacuation simulation using agent based modeling. Lastly, we will also descrive mechanism to contribute applications to the CyberGIS-Compute framework.
