A Caltech Library Service

Open Vocabulary Learning on Source Code with a Graph-Structured Cache

Cvitkovic, Milan and Singh, Badal and Anandkumar, Anima (2019) Open Vocabulary Learning on Source Code with a Graph-Structured Cache. Proceedings of Machine Learning Research, 97 . pp. 1475-1485. ISSN 1938-7228.

[img] PDF - Published Version
See Usage Policy.

[img] PDF - Submitted Version
See Usage Policy.

[img] PDF - Supplemental Material
See Usage Policy.


Use this Persistent URL to link to this item:


Machine learning models that take computer program source code as input typically use Natural Language Processing (NLP) techniques. However, a major challenge is that code is written using an open, rapidly changing vocabulary due to, e.g., the coinage of new variable and method names. Reasoning over such a vocabulary is not something for which most NLP methods are designed. We introduce a Graph-Structured Cache to address this problem; this cache contains a node for each new word the model encounters with edges connecting each word to its occurrences in the code. We find that combining this graph-structured cache strategy with recent Graph-Neural-Network-based models for supervised learning on code improves the models’ performance on a code completion task and a variable naming task — with over 100% relative improvement on the latter — at the cost of a moderate increase in computation time.

Item Type:Article
Related URLs:
URLURL TypeDescription Paper
Additional Information:© 2019 by the author(s). Proceedings of the 36th International Conference on Machine Learning, Long Beach, California, PMLR 97, 2019. Many thanks to Miltos Allamanis and Hyokun Yun for their advice and useful conversations.
Record Number:CaltechAUTHORS:20190327-085810844
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:94180
Deposited By: George Porter
Deposited On:28 Mar 2019 14:51
Last Modified:14 Feb 2020 23:05

Repository Staff Only: item control page