A Caltech Library Service

Context Generation Improves Open Domain Question Answering

Su, Dan and Patwary, Mostofa and Prabhumoye, Shrimai and Xu, Peng and Prenger, Ryan and Shoeybi, Mohammad and Fung, Pascale and Anandkumar, Anima and Catanzaro, Bryan (2022) Context Generation Improves Open Domain Question Answering. . (Unpublished)

Full text is not posted in this repository. Consult Related URLs below.

Use this Persistent URL to link to this item:


Closed-book question answering (QA) requires a model to directly answer an open-domain question without access to any external knowledge. Prior work on closed-book QA either directly finetunes or prompts a pretrained language model (LM) to leverage the stored knowledge. However, they do not fully exploit the parameterized knowledge. To address this issue, we propose a two-stage, closed-book QA framework which employs a coarse-to-fine approach to extract relevant knowledge and answer a question. Our approach first generates a related context for a given question by prompting a pretrained LM. We then prompt the same LM for answer prediction using the generated context and the question. Additionally, to eliminate failure caused by context uncertainty, we marginalize over generated contexts. Experimental results on three QA benchmarks show that our method significantly outperforms previous closed-book QA methods (e.g. exact matching 68.6% vs. 55.3%), and is on par with open-book methods that exploit external knowledge sources (e.g. 68.6% vs. 68.0%). Our method is able to better exploit the stored knowledge in pretrained LMs without adding extra learnable parameters or needing finetuning, and paves the way for hybrid models that integrate pretrained LMs with external knowledge.

Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription Paper
Su, Dan0000-0001-5746-9545
Fung, Pascale0000-0002-0628-7132
Anandkumar, Anima0000-0002-6974-6797
Catanzaro, Bryan0000-0003-0034-7728
Additional Information:This work was done when the first author was an intern at NVIDIA.
Record Number:CaltechAUTHORS:20221221-004708253
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:118552
Deposited By: George Porter
Deposited On:22 Dec 2022 18:41
Last Modified:02 Jun 2023 01:29

Repository Staff Only: item control page