Hardness-aware truth discovery in social sensing applications

Munira Syed

Abstract:

This paper develops a new principled framework to
solve a hardness-aware truth discovery problem in social sensing
applications. Social sensing has emerged as a new application
paradigm where a large crowd of social sensors (humans or
devices on their behalf) are recruited to or voluntarily report
observations about the physical environment at scale. These
observations may be either true or false, and hence are viewed
as binary claims. A fundamental problem in social sensing
applications lies in ascertaining the correctness of claims and
the reliability of data sources. We refer to this problem as
truth discovery. Significant efforts were made to address the
truth discovery problem, but an important dimension of the
problem has not been fully exploited: hardness of claims (how
challenging a claim is to be made). A common assumption
made in the previous work is that they assumed all claims are
of the same degree of hardness. However, in real world social
sensing applications, simply ignoring the hardness differences
between claims could easily lead to suboptimal truth discovery
results. In this paper, we develop a new hardness-aware truth
discovery scheme that explicitly considers different hardness
degrees of claims into a rigorous analytical framework. The new
truth discovery scheme solves a maximum likelihood estimation
problem to determine both the claim correctness and the source
reliability. We compare our hardness-aware scheme with the
state-of-the-art baselines through three real world case studies
(Baltimore Riots, Paris Attack and Oregon Shootings, all in
2015) using Twitter data feeds. The evaluation results showed
that our new scheme outperforms all compared baselines and
significantly improves the truth discovery accuracy in social
sensing applications.