A Caltech Library Service

Coupled Segmentation and Edge Learning via Dynamic Graph Propagation

Yu, Zhiding and Huang, Rui and Byeon, Wonmin and Liu, Sifei and Liu, Guilin and Breuel, Thomas and Anandkumar, Anima and Kautz, Jan (2021) Coupled Segmentation and Edge Learning via Dynamic Graph Propagation. In: 35th Conference on Neural Information Processing Systems (NeurIPS 2021). Neural Information Processing Foundation , La Jolla, CA, pp. 1-14. ISBN 9781713845393.

Full text is not posted in this repository. Consult Related URLs below.

Use this Persistent URL to link to this item:


Image segmentation and edge detection are both central problems in perceptual grouping. It is therefore interesting to study how these two tasks can be coupled to benefit each other. Indeed, segmentation can be easily transformed into contour edges to guide edge learning. However, the converse is nontrivial since general edges may not always form closed contours. In this paper, we propose a principled end-to-end framework for coupled edge and segmentation learning, where edges are leveraged as pairwise similarity cues to guide segmentation. At the core of our framework is a recurrent module termed as dynamic graph propagation (DGP) layer that performs message passing on dynamically constructed graphs. The layer uses learned gating to dynamically select neighbors for message passing using max-pooling. The output from message passing is further gated with an edge signal to refine segmentation. Experiments demonstrate that the proposed framework is able to let both tasks mutually improve each other. On Cityscapes validation, our best model achieves 83.7% mIoU in semantic segmentation and 78.7% maximum F-score in semantic edge detection. Our method also leads to improved zero-shot robustness on Cityscapes with natural corruptions (Cityscapes-C).

Item Type:Book Section
Related URLs:
URLURL TypeDescription
Huang, Rui0000-0003-0328-3862
Byeon, Wonmin0000-0002-4780-4749
Liu, Sifei0000-0002-6011-3686
Breuel, Thomas0000-0003-2417-4753
Anandkumar, Anima0000-0002-6974-6797
Kautz, Jan0000-0002-8830-429X
Additional Information:We thank the NVIDIA GPU Cloud (NGC) team for the computing support of this work. We also thank the anonymous reviewers and the other NVIDIA colleagues who helped to improve this work with discussions and constructive suggestions.
Record Number:CaltechAUTHORS:20221222-225518550
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:118599
Deposited By: George Porter
Deposited On:22 Dec 2022 23:18
Last Modified:22 Dec 2022 23:18

Repository Staff Only: item control page