A Caltech Library Service

Unsupervised RGB-to-Thermal Domain Adaptation via Multi-Domain Attention Network

Gan, Lu and Lee, Connor and Chung, Soon-Jo (2022) Unsupervised RGB-to-Thermal Domain Adaptation via Multi-Domain Attention Network. . (Unpublished)

[img] PDF - Submitted Version
See Usage Policy.


Use this Persistent URL to link to this item:


This work presents a new method for unsupervised thermal image classification and semantic segmentation by transferring knowledge from the RGB domain using a multi-domain attention network. Our method does not require any thermal annotations or co-registered RGB-thermal pairs, enabling robots to perform visual tasks at night and in adverse weather conditions without incurring additional costs of data labeling and registration. Current unsupervised domain adaptation methods look to align global images or features across domains. However, when the domain shift is significantly larger for cross-modal data, not all features can be transferred. We solve this problem by using a shared backbone network that promotes generalization, and domain-specific attention that reduces negative transfer by attending to domain-invariant and easily-transferable features. Our approach outperforms the state-of-the-art RGB-to-thermal adaptation method in classification benchmarks, and is successfully applied to thermal river scene segmentation using only synthetic RGB images. Our code is made publicly available at

Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription Paper ItemCode
Chung, Soon-Jo0000-0002-6657-3907
Additional Information:This work is funded by Ford Motor Company and in part by the Office of Naval Research.
Funding AgencyGrant Number
Ford Motor CompanyUNSPECIFIED
Office of Naval Research (ONR)UNSPECIFIED
Record Number:CaltechAUTHORS:20230316-225914015
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:120118
Deposited By: George Porter
Deposited On:17 Mar 2023 00:19
Last Modified:17 Mar 2023 00:19

Repository Staff Only: item control page