A Caltech Library Service

FLIP: Benchmark tasks in fitness landscape inference for proteins

Dallago, Christian and Mou, Jody and Johnston, Kadina E. and Wittmann, Bruce J. and Bhattacharya, Nicholas and Goldman, Samuel and Madani, Ali and Yang, Kevin K. (2021) FLIP: Benchmark tasks in fitness landscape inference for proteins. . (Unpublished)

[img] PDF (January 19, 2022) - Submitted Version
Creative Commons Attribution.

[img] PDF - Supplemental Material
Creative Commons Attribution.


Use this Persistent URL to link to this item:


Machine learning could enable an unprecedented level of control in protein engineering for therapeutic and industrial applications. Critical to its use in designing proteins with desired properties, machine learning models must capture the protein sequence-function relationship, often termed fitness landscape. Existing bench-marks like CASP or CAFA assess structure and function predictions of proteins, respectively, yet they do not target metrics relevant for protein engineering. In this work, we introduce Fitness Landscape Inference for Proteins (FLIP), a benchmark for function prediction to encourage rapid scoring of representation learning for protein engineering. Our curated tasks, baselines, and metrics probe model generalization in settings relevant for protein engineering, e.g. low-resource and extrapolative. Currently, FLIP encompasses experimental data across adeno-associated virus stability for gene therapy, protein domain B1 stability and immunoglobulin binding, and thermostability from multiple protein families. In order to enable ease of use and future expansion to new tasks, all data are presented in a standard format. FLIP scripts and data are freely accessible at

Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription Paper ItemFitness Landscape Inference for Proteins ItemCode
Dallago, Christian0000-0003-4650-6181
Johnston, Kadina E.0000-0002-2214-3534
Wittmann, Bruce J.0000-0001-8144-9157
Yang, Kevin K.0000-0001-9045-6826
Additional Information:The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY 4.0 International license. Version 1 - November 11, 2021; Version 2 - January 19, 2022. The authors thank Jeffrey Spencer, Sam Sinai, Sam Bowman, Roshan Rao and Debora Marks for ideas and discussions that helped us improve our work. The authors would also like to thank Helix and Murphy for careful attention to the manuscript. C.D. acknowledges support from the Bundesministerium für Bildung und Forschung (BMBF) – Project numbers: 01IS17049 and 031L0168. K.E.J. and B.J.W. acknowledge the NSF Division of Chemical, Bioengineering, Environmental and Transport Systems (1937902). N.B. was supported in part by NIH grant R35-GM134922 and by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration. S.G. thanks the MIT Machine Learning for Pharmaceutical Discovery and Synthesis Consortium for supporting this work. Competing Interest Statement: KKY was previously employed by Generate Biomedicines.
Funding AgencyGrant Number
Bundesministerium für Bildung und Forschung (BMBF)01IS17049
Bundesministerium für Bildung und Forschung (BMBF)031L0168
Department of Energy (DOE)17-SC-20-SC
Massachusetts Institute of Technology (MIT)UNSPECIFIED
Record Number:CaltechAUTHORS:20211112-205605548
Persistent URL:
Official Citation:FLIP: Benchmark tasks in fitness landscape inference for proteins. Christian Dallago, Jody Mou, Kadina E Johnston, Bruce Wittmann, Nicholas Bhattacharya, Samuel Lucas Goldman, Ali Madani, Kevin K Yang. bioRxiv 2021.11.09.467890; doi:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:111857
Deposited By: Tony Diaz
Deposited On:12 Nov 2021 21:07
Last Modified:21 Jan 2022 17:42

Repository Staff Only: item control page