Kittenplon, Yair and Lavi, Inbal and Fogel, Sharon and Bar, Yarin and Manmatha, R. and Perona, Pietro (2022) Towards Weakly-Supervised Text Spotting using a Multi-Task Transformer. . (Unpublished) https://resolver.caltech.edu/CaltechAUTHORS:20220224-200946567
![]() |
PDF
- Submitted Version
See Usage Policy. 12MB |
Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20220224-200946567
Abstract
Text spotting end-to-end methods have recently gained attention in the literature due to the benefits of jointly optimizing the text detection and recognition components. Existing methods usually have a distinct separation between the detection and recognition branches, requiring exact annotations for the two tasks. We introduce TextTranSpotter (TTS), a transformer-based approach for text spotting and the first text spotting framework which may be trained with both fully- and weakly-supervised settings. By learning a single latent representation per word detection, and using a novel loss function based on the Hungarian loss, our method alleviates the need for expensive localization annotations. Trained with only text transcription annotations on real data, our weakly-supervised method achieves competitive performance with previous state-of-the-art fully-supervised methods. When trained in a fully-supervised manner, TextTranSpotter shows state-of-the-art results on multiple benchmarks.
Item Type: | Report or Paper (Discussion Paper) | ||||||
---|---|---|---|---|---|---|---|
Related URLs: |
| ||||||
ORCID: |
| ||||||
Record Number: | CaltechAUTHORS:20220224-200946567 | ||||||
Persistent URL: | https://resolver.caltech.edu/CaltechAUTHORS:20220224-200946567 | ||||||
Usage Policy: | No commercial reproduction, distribution, display or performance rights in this work are provided. | ||||||
ID Code: | 113607 | ||||||
Collection: | CaltechAUTHORS | ||||||
Deposited By: | George Porter | ||||||
Deposited On: | 25 Feb 2022 00:22 | ||||||
Last Modified: | 25 Feb 2022 00:22 |
Repository Staff Only: item control page