Denoising Radar Satellite Images with Python Has Never Been So Easy

Presentation of the latest release of deepdespeckling

Synthetic aperture radar (SAR) images are widely use in a large variety of sectors (aerospace, military, meteorology, etc.). The problem is this kind of images suffer from noise in their raw format. While these images are also usually heavy files, the task of denoising it efficiently appears to be both challenging from a scientific perspective and very useful in the real world.

In this Towards Data Science article, we presented deepdespeckling, an open-source python package enabling to despeckle synthetic aperture radar (SAR) images using a novel deep learning based method.

We are happy to announce that we have released a new version of deepdespeckling, enabling to use both MERLIN and SAR2SAR methods for despeckling radar satellite images.

A quick reminder on satellite images

There are two big categories of satellite images :

• Optical images : the ones we are used to see when we watch a weather forecast for example. These images are taken by optical sensors.
  While these images generally provide a high level of detail, they encounter at least two significant challenges in capturing Earth’s intricacies: the limitations posed by nighttime conditions and adverse weather.
• Radar images : while optical systems rely on the sunlight (the sensor is passive), radars send an electromagnetic wave and measure the component backscattered by the objects on the ground (the sensor is active). radar sensors can acquire data at any time of the day and with any meteorological conditions, as the wavelength of the transmitted wave allows it to penetrate clouds. They however encounter an intrinsic issue : speckle noise.

What is speckle noise ?

Speckle is a granular interference due to bouncing properties of emitted radio waves that degrades the quality of images and therefore their interpretability with a human eye.

How to get rid of it

Several methods exist, but deep learning has brought significant improvements for this task. Emanuele Dalsasso, Loïc Denis and Florence Tupin developed two deep learning based methods for despeckling SAR images :

• MERLIN (coMplex sElf-supeRvised despeckLINg) : a self-supervised strategy based on the separation of the real and imaginary parts of single-look complex SAR images that we presented in the previous Towards Data Science article
• SAR2SAR : Multi-temporal time series are leveraged in order to train neural network to restore SAR images by only looking at noisy acquisitions. This method is part of the new features of the latest release of deepdespeckling. Hence, we will focus on this method in this article

SAR2SAR

Just as MERLIN, SAR2SAR also draws inspiration from the noise2noise algorithm, which showed that it is possible to train a model to denoise without looking at noise-free examples. This feature is of particular importance in SAR despeckling, as speckle-free acquisition do not exist.

SAR2SAR builds on the assumption that two images acquired over the same area at different times are corrupted by two uncorrelated speckle realizations, matching with the hypothesis allowing the application of the noise2noise principle. This allows to develop a model to remove speckle from Ground Range Detected (GRD) SAR images, which are only available in amplitude (the phase is suppressed during the detection step) and thus MERLIN cannot be used on such data. Temporal acquisitions are leveraged to generate a dataset containing independent speckle realisations of the same scene (a change compensations strategy relying on a pre-trained model is used to ensure that the temporal acquisitions only differ for the speckle component).

Once the model is trained, during inference SAR2SAR requires a single GRD image and can be effectively deployed to suppress speckle from Sentinel-1 GRD SAR images.

SAR images acquisition

Different acquisition modes exist depending on the compromise between the illuminated scene (the swath) and and the image resolution. Each acquisition mode thus produces images having a different resolution, thus the appearance of objects is specific to each acquisition mode.

For this reason, a model specific for each modality must be developed. Given the simplicity of application of MERLIN, which requires single SAR images, datasets for each specific modality can be seamlessly collected. We have trained MERLIN on the following images:

• TerraSAR-X images acquired in Stripmap mode
• TerraSAR-X images acquired in HighResolution SpotLight mode
• Sentinel-1 images acquired in TOPS mode

deepdespeckling package usage

Package installation

Before installing deepdespeckling, make sure to install gdal dependancies, it can be done using conda with the following command :

conda install -c conda-forge gdal

Then you can install the package this way :

pip install deepdespeckling

Despeckle one image with MERLIN

To despeckle SAR images using MERLIN, images need to be in .cos or .npy format.

Two parameters have to be set:

• model_name : “spotlight” for SAR images retrieved with spotlight mode, “stripmap” for SAR images retrieved with stripmap mode or “Sentinel-TOPS” for images retrieved with TOPS mode
• symetrise: during the preprocessing steps of the noisy image for MERLIN, the real and the imaginary parts are “symetrised” (to match the theoretical assumptions of MERLIN). To skip this step, the symetrise parameter can be set to False

from deepdespeckling.utils.load_cosar import cos2mat
from deepdespeckling.utils.constants import PATCH_SIZE, STRIDE_SIZE
from deepdespeckling.merlin.merlin_denoiser import MerlinDenoiser

# Path to one image (cos or npy file)
image_path="path/to/cosar/image"
# Model name, can be "spotlight", "stripmap" or "Sentinel-TOPS"
model_name = "spotlight"
symetrise = True

image = cos2mat(image_path).astype(np.float32)

denoiser = MerlinDenoiser(model_name=model_name, symetrise=symetrise)
denoised_image = denoiser.denoise_image(image, patch_size=PATCH_SIZE, stride_size=STRIDE_SIZE)

This snippet of code will store the despeckled image in a numpy array in the denoised_image variable.

Despeckle one image with SAR2SAR

To despeckle SAR images using SAR2SAR, images need to be in .tiff or .npy format.

from deepdespeckling.utils.load_cosar import cos2mat
from deepdespeckling.utils.constants import PATCH_SIZE, STRIDE_SIZE
from deepdespeckling.sar2sar.sar2sar_denoiser import Sar2SarDenoiser

# Path to one image (tiff or npy file)
image_path="path/to/cosar/image"

# Works exactly the same as with MERLIN
image = cos2mat(image_path).astype(np.float32)

# Denoise the image with SAR2SAR
denoiser = Sar2SarDenoiser()
denoised_image = denoiser.denoise_image(image, patch_size=PATCH_SIZE, stride_size=STRIDE_SIZE)

Despeckle a set of images using MERLIN or SAR2SAR

For both MERLIN and SAR2SAR, you can choose between 3 different functions to despeckle a set of SAR images contained in a folder :

• despeckle to despeckle full size images
• despeckle_from_coordinates to despeckle a sub-part of the images defined by some coordinates
• despeckle_from_crop to despeckle a sub-part of the images defined using a crop tool

Despeckle fullsize images

from deepdespeckling.despeckling import despeckle

# Path to a folder of several images 
# images have to be in .tiff or .npy formats if using sar2sar 
# images have to be in .cos or .npy formats is using merlin ("spotlight", "stripmap" or "Sentinel-TOPS")
image_path="path/to/cosar/image"
# Folder where results are stored
destination_directory="path/where/to/save/results"

# Can be "sar2sar", "spotlight' or "stripmap"
model_name = "spotlight"
# symetrise parameter if using "spotlight", "stripmap" or "Sentinel-TOPS" (harmless if using "sar2sar")
symetrise = True

despeckle(image_path, destination_directory, model_name=model_name, symetrise=symetrise)

The despeckle function will create several folders in the destination_directory :

• processed_images: the npy files (numpy array conversion) of the raw images stored in the folder defined in image_path.
• noisy:the preprocessed noisy images in both .npy and .png formats
• denoised: the denoised images in both .npy and .png formats

Despeckle parts of images using custom coordinates

from deepdespeckling.despeckling import despeckle_from_coordinates

# Path to a folder of several images 
# images have to be in .tiff or .npy formats if using sar2sar 
# images have to be in .cos or .npy formats is using merlin ("spotlight", "stripmap" or "Sentinel-TOPS")
image_path="path/to/cosar/image"
# Folder where results are stored
destination_directory="path/where/to/save/results"
# Example of coordinates of the subparts of the images to be despeckled
coordinates_dictionnary = {'x_start':2600,'y_start':1000,'x_end':3000,'y_end':1200}

# Can be "sar2sar", "spotlight", "stripmap" or "Sentinel-TOPS"
model_name = "spotlight"
# symetrise parameter if using "spotlight", "stripmap" or "Sentinel-TOPS" (harmless if using "sar2sar")
symetrise = True

despeckle_from_coordinates(image_path, coordinates_dict, destination_directory, 
                            model_name=model_name, symetrise=symetrise)

Thedespeckle_from_coordinates function will create the same folders as thedespeckle function, with images croped with the specified coordinates.

Despeckle parts of images using a crop tool

from deepdespeckling.merlin.inference.despeckling import despeckle_from_crop

# Path to a folder of several images 
# images have to be in .tiff or .npy formats if using sar2sar 
# images have to be in .cos or .npy formats is using merlin ("spotlight", "stripmap" or "Sentinel-TOPS")
image_path="path/to/cosar/image"
# Folder where results are stored
destination_directory="path/where/to/save/results"

# If True it will crop a 256*256 image from the position of your click
# If False you will draw free-handly the area of your interest
fixed = True
# Can be "sar2sar", "spotlight", "stripmap" or "Sentinel-TOPS"
model_name = "spotlight"
# symetrise parameter if using "spotlight""stripmap" or "Sentinel-TOPS" (harmless if using "sar2sar")
symetrise = True

despeckle_from_crop(image_path, destination_directory, model_name=model_name, fixed=fixed, symetrise=symetrise)

Thedespeckle_from_crop function will first launch the crop tool : just select an area and press “q” when you are satisfied with the crop

Then, the despeckle_from_crop function will create :

• The same folders as thedespeckle function, with images cropped using the crop tool
• cropping_coordinates.txt file containing the coordinates of the selected crop

Going further

Now you know to use deepdespeckling, to understand further more how it works, you can check the github repository. We also provide a sphinx documentation available here.

Feel free to contact me for any questions and feedback !

Authors

• Hadrien Mariaccia
• Emanuele Delsasso

Unless otherwise noted, all images are by the authors

Contact

Don’t hesitate to contact me if you have any questions.

To know more about Hi! PARIS and its Engineering Team:

• Hi! PARIS
• Hi! PARIS Engineering Team

Denoising Radar Satellite Images with Python Has Never Been So Easy was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.