Workshop at NeurIPS 2020

September 18, 2020

We are happy to announce that the workshop “Crowd Science Workshop: Remoteness, Fairness, and Mechanisms as Challenges of Data Supply by Humans for Automation” will be conducted at the world's top conference on machine learning — NeurIPS 2020.

The expansion of AI involves several hot issues related to the workforce (data supply demands, disappearing professions, and undesirable working conditions) that can be overcome with the right crowdsourcing methodology. We focus on remoteness, fairness, and economic mechanisms as the critical aspects of successful data collection and labeling. 

Remoteness. Data labeling requesters (data consumers for ML systems) are skeptical of the effectiveness and efficiency of remote work. They need trustworthy quality control techniques and ways to guarantee reliable results on time. Crowdsourcing is a viable solution for effective remote work. However, in spite of its rapid growth and the body of literature available on the topic, crowdsourcing is still in its infancy and, to a large extent, is considered an art. Standard approaches lack clear guidelines and accepted practices for both the requester and the performers (also known as workers), which holds crowdsourcing back from achieving its potential. Our intention is to end this trend and achieve a breakthrough in this direction. 

Fairness. Crowd workers (data suppliers) are skeptical of the availability and choice of tasks. They need fair and ethical task distribution, reasonable compensation, and growth opportunities. We believe that the working environment (a crowdsourcing platform) can help solve these issues by offering flexible work hours and the freedom to choose and switch tasks, as well as by acting fairly and ethically in task distribution. We also aim to address bias in task design and execution that can skew results in ways that are not anticipated by data requesters. 

Since quality, fairness, and growth opportunities for performers are central to our workshop, we will invite a diverse group of performers from a global public crowdsourcing platform to our panel-led discussion. 

Mechanisms. Matchmakers (the working environment, usually represented by a crowdsourcing platform) may doubt the effectiveness of economic drivers behind their two-sided market. They need a mechanism designed to guarantee proper incentives for both sides: flexibility and fairness for workers, with quality and efficiency for data requesters. Our emphasis is on economic mechanisms as the key to successfully address issues of remoteness and fairness. Our intention is to deepen the interaction of communities that work on mechanisms and crowdsourcing. 

We invite you to participate in our workshop (registration for the conference is required). In addition, anyone can present their research - just submit an application. Call for Papers is until October 9, 2020.