Online tutorial “Improving Web Ranking with Human-in-the-Loop: Methodology, Scalability, Evaluation” at The Web Conference 2021
In our hands-on tutorial, we present a systematic view on using Human-in-the-Loop to obtain scalable offline evaluation processes and, in particular, high-quality relevance judgements.
April 13, Online
“Crowd Science Workshop: Remoteness, Fairness, and Mechanisms as Challenges of Data Supply by Humans for Automation” at NeurIPS 2020
We will gather the world's best experts to discuss the key issues of preparing labeled data for machine learning. We will focus on remoteness, fairness, and mechanisms in the context of crowdsourcing for data collection and labeling.
December 11–12, Online
Online tutorial "Efficient Data Annotation for Self-Driving Cars via Crowdsourcing on a Large-Scale" at CVPR 2020
We will present a data processing pipeline required for the self-driving cars to learn how to behave autonomously on the roads. We will also demonstrate how data annotation constitutes a crucial part that makes the learning process effective. During our tutorial the participants will practice in launching the projects from the pipeline on one of the largest crowdsourcing marketplaces.
June 15, Seattle, USA
Online tutorial "Crowdsourcing Practice for Efficient Data Labeling: Aggregation, Incremental Relabeling, and Pricing" at SIGMOD 2020
We will make an introduction to data labeling via public crowdsourcing marketplaces and will present the key components of efficient label collection. This will be followed by a practice session, where participants will launch their label collection project on one of the largest crowdsourcing marketplaces.
June 14, Portland, USA
Tutorial "Practice of Efficient Data Collection via Crowdsourcing: Aggregation, Incremental Relabelling, and Pricing" at WSDM 2020
In this tutorial, we present a portion of unique industry experience in efficient data labeling via crowdsourcing shared by leading researchers and engineers from Yandex.
February 3, Houston, USA