Follow our latest advances in machine learning.

Generative models

Computer vision

Graph machine learning

Distributional shift

Natural language processing 

Nearest neighbor search

Uncertainty estimation 

Large-scale machine learning

Machine learning theory

Neural algorithmic reasoning

Tabular data

Distributed ML

Gradient boosting

Speculative and parallel decoding

Blog

Yandex Research Blog

In 2024, Yandex Research has evolved into several dedicated research groups, each focused on a specific scientific area: Generative Computer Vision, Efficient Large Language Models (LLMs), Graph Neural Networks, and Tabular Deep Learning.

We presented 17 papers at major conferences, including NeurIPS, ICLR, ICML, CVPR, and ACL. Several papers were co-authored with researchers from prominent global institutions, including IST Austria, KAUST Saudi Arabia, Carnegie Mellon University, Meta AI, and Together AI.

Yandex Research in 2024

Eight papers by the Yandex Research team and our collaborators have been accepted for publication at the Conference on Neural Information Processing Systems (NeurIPS 2024).

Papers accepted to NeurIPS 2024

large_dfs_16_9.png

medium_dfs_16_9.png

small_dfs_16_9.png

thumbnail_dfs_16_9.png

dfs_16_9.png

Algorithmic reasoning focuses on building models that can execute classic algorithms. It allows one to combine the advantages of neural networks with theoretical guarantees of algorithms.

Graphs are a natural way to represent data from various domains such as social networks, molecules, text, code, etc. We develop and analyze algorithms for graph-structured data.

In this work, we achieve perfect neural execution of several algorithms by forcing the node and edge representations to be from a fixed finite set. Also, the proposed architectural choice allows us to prove the correctness of the learned algorithms for any test data.

Discrete neural algorithmic reasoning

Today, training most powerful models often takes significant resources. Our research aims to make large-scale training more efficient and accessible to the entire machine learning community.

Language is one of the key forms of communication. We study methods of language representation and understanding to simplify human-computer interactions.

Modern LLMs are autoregressive models that generate one token at a time, which is inefficient on parallel hardware. These works accelerate generation by processing multiple tokens per forward pass.


We introduce SpecExec, a speculative decoding method that delivers up to 20 tokens per iteration and up to 15x speedups in offloading settings. This enables usable LLM inference speeds for users with limited GPU VRAM sizes, who have had to use offloading or settle for lower quality models until now. 

SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices

Three papers by the Yandex Research team (shown in bold below) and our collaborators have been accepted for publication at the International Conference on Machine Learning (ICML 2024).

Blog

Posts

Yandex Research in 2024

Papers accepted to NeurIPS 2024

Discrete neural algorithmic reasoning

SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices

Papers accepted to ICML 2024

Filter by: