Nathan Hahn


Alloy: Clustering with Crowds and Computation

Joseph Chang, Nathan Hahn, Niki Kittur

CHI 2016Honorable MentionPDF


Crowdsourced clustering approaches present a promising way to harness deep semantic knowledge for clustering complex information. However, existing approaches have difficulties supporting the global context needed for workers to generate meaningful categories, and are costly because all items require human judgments. We introduce Alloy, a hybrid approach that combines the richness of human judgments with the power of machine algorithms. Alloy supports greater global context through a new sample and search crowd pattern which changes the crowd’s task from classifying a fixed subset of items to actively sampling and querying the entire dataset. It also improves efficiency through a two phase process in which crowds provide examples to help a machine cluster the head of the distribution, then classify low-confidence examples in the tail. To accomplish this, Alloy introduces a modular cast and gather approach which leverages a machine learning backbone to stitch together different types of judgment tasks.


Chang, J. C., Kittur, A., & Hahn, N. (2016). Alloy. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI  ’16. doi:10.1145/2858036.2858411


	doi = {10.1145/2858036.2858411},
	url = {},
	year = 2016,
	publisher = {{ACM} Press},
	author = {Joseph Chee Chang and Aniket Kittur and Nathan Hahn},
	title = {Alloy},
	booktitle = {Proceedings of the 2016 {CHI} Conference on Human Factors in Computing Systems - {CHI} {\textquotesingle}16}