David vs. Goliath: The Art of Leaderboarding in the Era of Extreme-Scale Neural Models

Scale appears to be the winning recipe in today’s leaderboards. And yet, extreme-scale neural models are still brittle to make errors that are often nonsensical and even counterintuitive. In this talk, Yejin Choi will argue for the importance of knowledge, especially common-sense knowledge, as well as inference-time reasoning algorithms, and demonstrate how smaller models developed in academia can still have an edge over larger industry-scale models, if powered with knowledge and/or reasoning algorithms.

Abstract

Scale appears to be the winning recipe in today’s leaderboards. And yet, extreme-scale neural models are still brittle to make errors that are often nonsensical and even counterintuitive. In this talk, I will argue for the importance of knowledge, especially common-sense knowledge, as well as inference-time reasoning algorithms, and demonstrate how smaller models developed in academia can still have an edge over larger industry-scale models, if powered with knowledge and/or reasoning algorithms. First, I will introduce “symbolic knowledge distillation”, a new framework to distill larger neural language models into smaller common-sense models, which leads to a machine-authored KB that wins, for the first time, over a human-authored KB in all criteria: scale, accuracy, and diversity. Next, I will highlight how we can make better lemonade out of neural language models by shifting our focus to unsupervised, inference-time reasoning algorithms. I will demonstrate how unsupervised models powered with algorithms can match or even outperform supervised approaches on hard reasoning tasks such as nonmonotonic reasoning (such as counterfactual and abductive reasoning), or complex language generation tasks that require logical constraints. Finally, I will introduce a new (and experimental) conceptual framework, Delphi, toward machine norms and morality, so that the machine can learn to reason that “helping a friend” is generally a good thing to do, but “helping a friend spread fake news” is not.

Biography

Yejin Choi is a Professor at Paul G. Allen School of Computer Science & Engineering at University of Washington and Allen Institute for Artificial Intelligence (AI2). Her primary research interests are in the fields of Natural Language Processing, Machine Learning and Artificial Intelligence, with broader interests in Computer Vision and Digital Humanities.

Additional Information:

Join via Zoom.