Raindrop-aware gan: Unsupervised learning for raindrop-contaminated coastal video enhancement

Jinah Kim, Dong Huh, Taekyung Kim, Jaeil Kim, Jeseon Yoo, Jae Seol Shim

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

We propose an unsupervised network with adversarial learning, the Raindrop-aware GAN, which enhances the quality of coastal video images contaminated by raindrops. Raindrop removal from coastal videos faces two main difficulties: converting the degraded image into a clean one by visually removing the raindrops, and restoring the background coastal wave information in the raindrop regions. The components of the proposed network—a generator and a discriminator for adversarial learning—are trained on unpaired images degraded by raindrops and clean images free from raindrops. By creating raindrop masks and background-restored images, the generator restores the background information in the raindrop regions alone, preserving the input as much as possible. The proposed network was trained and tested on an open-access dataset and directly collected dataset from the coastal area. It was then evaluated by three metrics: the peak signal-to-noise ratio, structural similarity, and a naturalness-quality evaluator. The indices of metrics are 8.2% (+2.012), 0.2% (+0.002), and 1.6% (−0.196) better than the state-of-the-art method, respectively. In the visual assessment of the enhanced video image quality, our method better restored the image patterns of steep wave crests and breaking than the other methods. In both quantitative and qualitative experiments, the proposed method more effectively removed the raindrops in coastal video and recovered the damaged background wave information than state-of-the-art methods.

Original languageEnglish
Article number3461
Pages (from-to)1-18
Number of pages18
JournalRemote Sensing
Volume12
Issue number20
DOIs
StatePublished - 2 Oct 2020

Keywords

  • Background information recovery
  • Coastal video enhancement
  • Generative adversarial network
  • Raindrop removal
  • Unsupervised learning

Fingerprint

Dive into the research topics of 'Raindrop-aware gan: Unsupervised learning for raindrop-contaminated coastal video enhancement'. Together they form a unique fingerprint.

Cite this