Introduction to Computer Vision for Climate Change

I was born and raised in Ecuador. In this country, weather and climate shape our lives. For example, our energy supply relies on sufficient rainfall for hydroelectric power. As a child, I remember having continuous blackouts. Unfortunately, Ecuador has not been resilient. At the time of writing this article, we are experiencing blackouts again. Paradoxically, El Niño Southern Oscillation brings us flooding every year. I love hiking, and with great sadness, I saw how our glaciers have retreated.

Ten years ago, I decided to study for a PhD in meteorology. Climate change and its implications troubled me. It is a daunting challenge that humanity faces in this century. There has been enormous progress in our scientific understanding of this problem. But we still need more action.

When I started my PhD, few researchers used artificial intelligence (AI) techniques. Nowadays, there is a consensus that harnessing the potential of AI can make a difference. In particular, in mitigating and adapting to climate change.

ML and in particular computer vision (CV) empower us to make sense of the massive amounts of available data. This power will allow us to take action. Uncovering hidden patterns in visual data (eg. satellite data) is a critical task in tackling climate change.

This article introduces CV and its intersection with climate change. It is the first of a series on this topic. The article has five sections. First, it presents an introduction. Next, the article defines some basic concepts related to CV. Then, it explores the capabilities of CV to tackle climate change with case studies. After that, the article discusses challenges and future directions. Finally, a summary provides an overview.

Understanding Computer Vision

CV uses computational methods to learn patterns from images. Earth Observation (EO) relies mainly on satellite images. Thus, CV is a well-suited tool for climate change analysis. To understand climate patterns from images, several techniques are necessary. Some of the most important are classification, object detection, and segmentation.

Classification: involves categorizing (single) images based on predefined classes (single labels). Fire detection and burned area mapping use image classification techniques on satellite images. These images provide spectral signatures linked to burned vegetation. Using these unique patterns researchers can track the impact of wildfires.

Object detection: comprises locating objects in an area of interest. The track of hurricanes and cyclones uses this technique. Detecting its cloud patterns helps to mitigate their impact in coastal zones.

Image segmentation: assigns a class to each pixel in an image. This technique helps to identify regions and their boundaries. Segmentation is also referred to as “semantic segmentation”. Since each region (target class) receives a label its definition includes “semantic”. For example, tracking a glacier’s retreat uses this technique. Segmenting satellite images from glaciers allows for tracking their changes. For instance, monitoring glacier’s extent, area, and volume over time.

This section provided some examples of CV in action to tackle climate change. The following section will analyze them as case studies.

Case Study 1: Wildfire detection

Climate change has several implications for wildfires. For example, increasing the likelihood of extreme events. Also, extending the timeframe of fire seasons. Likewise, it will exacerbate fire intensity. Thus, investing resources in innovative solutions to prevent catastrophic wildfires is imperative.

This type of research depends on the analyses of images for early detection of wildfires. ML methods, in general, proved to be effective in predicting these events.

However, advanced AI deep learning algorithms yield the best results. An example of these advanced algorithms is Neural Networks (NNs). NNs are an ML technique inspired by human cognition. This technique relies on one or more convolutional layers to detect features.

Convolutional Neural Networks (CNN) are popular in Earth Science applications. CNN shows the greatest potential to increase the accuracy of fire detection. Several models use this algorithm, such as VGGNet, AlexNet, or GoogleNet. These models present improved accuracy in CV tasks.

Fire detection through CV algorithms requires image segmentation. Yet, before segmenting the data, it needs preprocessing. For instance, to reduce noise, normalize values, and resize. Next, the analysis labels pixels that represent fire. Thus distinguishing them from other image information.

Case Study 2: Cyclone Tracking

Climate change will increase the frequency and intensity of cyclones. In this case, a massive amount of data is not processed by real-time applications. For instance, data from models, satellites, radar, and ground-based weather stations. CV demonstrates to be efficient in processing these data. It has also reduced the biases and errors linked with human intervention.

For example, numerical weather prediction models use only 3%–7% of data. In this case, observations from Geostationary Operational Environmental Satellites (GOES). The data assimilation processes use even less of these data. CNN models select among this vast quantity of images the most relevant observations. These observations refer to cyclone-active (or soon-to-be active) regions of interest (ROI).

Identifying this ROI is a segmentation task. There are several models used in Earth Sciences to approach this problem. Yet, the U-Net CNN is one of the most popular choices. The model design relates to medical segmentation tasks. But it has proven useful in solving meteorological problems as well.

Case Study 3: Tracking Glacial Retreat

Glaciers are thermometers of climate change. The effects of climate variations on glaciers are visual (retreat of outlines). Thus, they symbolize the consequences of climate variability and change. Besides the visual impacts, the glacier retreat has other consequences. For example, adverse effects on water resource sustainability. Destabilization of hydropower generation. Affecting drinking water quality. Reductions in agricultural production. Unbalancing ecosystems. On a global scale, even the increase in sea level threatens coastal regions.

The process of monitoring glaciers used to be time-consuming. The interpretation of satellite images needs experts to digitalize and analyze them. CV can help to automate this process. Additionally, computer vision can make the process more efficient. For example, allowing the incorporation of more data into the modeling. CNN models such as GlacierNet harness the power of deep learning to track glaciers.

There are several techniques to detect glacier boundaries. For example, segmentation, object detection, and also edge detection. CV can perform even more complex tasks. Comparing glacier images over time is one example. Likewise, determining the velocity of movement of glaciers and even their thickness. These are powerful tools to track glacier dynamics. These processes can extract valuable information for adaptation purposes.

Challenges and Future Directions

There are particular challenges in tackling climate change using CV. Discussing each of them may need an entire book. However, the aim here is modest. I will attempt to bring them to the table for a reference.

• Data complexity: The need, and the inherent complexity, of using many sources of data. For example, satellite and aerial imagery, lidar data, and ground-based sensors. Data fusion is an evolving technique that attempts to address this challenging issue.
• Model interpretability: a current challenge is developing hybrid models. It means reconciling a statistical data-driven model with a physical one. The interpretability of CV algorithms increases incorporating our knowledge of the climate system. Thus, these models excel in fitting complex functions. But also should provide an understanding of the underlying causal relations.
• Labeled samples: The availability of high-quality labeled samples. These samples should be specific to EO problems to train CV models. Generating them is a time-consuming and costly task. Addressing this challenge is an active area of research.
• Ethics: Is a challenge to incorporate ethical considerations in AI development. Privacy, fairness, and accountability play a key role in ensuring trust with stakeholders. Considering environmental justice is also a sound strategy in the context of climate change.

Summary

CV is a powerful tool to tackle climate change. From detecting wildfires to tracking cyclone formation and glacier retreats. CV is transforming how to monitor, predict, and project climate impacts. The study of these impacts relies on CV techniques. For example, classification, object detection, and segmentation. Finally, several challenges arise in the intersection between CV and climate change. For instance, managing multiple sources of data. Enhancing the interpretability of machine learning models. Generating high-quality labeled samples to train CV models. And incorporating ethical considerations when designing an AI system. A subsequent article will present a guide to collecting and curating image datasets. In particular, those relevant to climate change.

References

• Kumler-Bonfanti, C., Stewart, J., Hall, D., & Govett, M. (2020). Tropical and extratropical cyclone detection using deep learning. Journal of Applied Meteorology and Climatology, 59(12), 1971–1985.
• Maslov, K. A., Persello, C., Schellenberger, T., & Stein, A. (2024). Towards Global Glacier Mapping with Deep Learning and Open Earth Observation Data. arXiv preprint arXiv:2401.15113.
• Moumgiakmas, S. S., Samatas, G. G., & Papakostas, G. A. (2021). Computer vision for fire detection on UAVs — From software to hardware. Future Internet, 13(8), 200.
• Rolnick, D., Donti, P. L., Kaack, L. H., Kochanski, K., Lacoste, A., Sankaran, K., … & Bengio, Y. (2022). Tackling climate change with machine learning. ACM Computing Surveys (CSUR), 55(2), 1–96.
• Tuia, D., Schindler, K., Demir, B., Camps-Valls, G., Zhu, X. X., Kochupillai, M., … & Schneider, R. (2023). Artificial intelligence to advance Earth observation: a perspective. arXiv preprint arXiv:2305.08413.

Introduction to Computer Vision for Climate Change was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.