Andrey Malinin

Neil Band

Yarin Gal

Mark J. F. Gales

Alexander Ganshin

German Chesnokov

Alexey Noskov

Andrey Ploskonosov

Liudmila Prokhorenkova

Ivan Provilkov

Vatsal Raina

Vyas Raina

Denis Roginskiy

Mariya Shmatova

Panos Tigas

Boris Yangel

The Shifts Dataset contains curated and labeled examples of real, 'in-the-wild' distributional shifts across three large-scale tasks. Specifically, it contains tabular weather prediction, machine translation, and vehicle motion prediction tasks' data used in Shifts Challenge 2021. Dataset shift is ubiquitous in all of these tasks and modalities.

Machine learning (ML) is essential for innovations such as autonomous vehicle technology (AV), weather forecasting and machine translation, however, the application of ML creates its own challenges. For instance, the mismatch between training and deployment data, known as distributional shift, can cause a degradation in model performance as the degree of shift increases. This is especially important to be aware of in applications with strict safety requirements.

Shifts Challenge at NeurIPS 2021

One of the biggest challenges in applying machine learning to real-world tasks is the mismatch between training and deployment data, known as distributional shift. This mismatch can cause a degradation in model performance as the degree of shift increases. Distributional shift is ubiquitous in machine learning and is especially important to be aware of in applications with strict safety requirements.

Outcomes of the Shifts Challenge

Distributional shift is the mismatch between training and deployment data that is ubiquitous in the real-world. Studying this phenomenon can enable safer and more reliable ML systems.

Distributional shift

Uncertainty estimation enables detecting when ML models make mistakes. This is of critical importance in high risk machine learning applications, such as autonomous vehicle and medical ML.

Uncertainty estimation 

Tabular data involves two-dimensional tables with objects (rows) and features (columns), which are used in numerous applied tasks such as classification, regression, ranking and many others.

Tabular data

Language barriers hinder global communication and access to worldwide knowledge. By improving machine translation systems, we hope to facilitate the exchange of culture and information.

Machine translation

Language is one of the key forms of communication. We study methods of language representation and understanding to simplify human-computer interactions.