Classification-oriented dawid skene model for transferring intelligence from crowds to machines

When a crowdsourcing approach is used to assist the classification of a set of items, the main objective is to classify this set of items by aggregating the worker-provided labels. A secondary objective is to assess the workers’ skill levels in this process. A classical model that achieves both objectives is the famous Dawid-Skene model. In this paper, we consider a third objective in this context, namely, to learn a classifier that is capable of labelling future items without further assistance of crowd workers. By extending the Dawid-Skene model to include the item features into consideration, we develop a Classification-Oriented Dawid Skene (CODS) model, which achieves the three objectives simultaneously. The effectiveness of CODS on this three dimensions of the problem space is demonstrated experimentally.